[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 466878196 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002380] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004014] kvm-guest: setup PV IPIs [ 0.007535] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009013] pid_max: default: 32768 minimum: 301 [ 0.010128] LSM: Security Framework initializing [ 0.011058] Yama: becoming mindful. [ 0.012040] SELinux: Initializing. [ 0.013079] *** VALIDATE selinux *** [ 0.021756] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025759] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026147] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027119] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028093] *** VALIDATE tmpfs *** [ 0.029437] *** VALIDATE proc *** [ 0.030234] *** VALIDATE cgroup *** [ 0.031011] *** VALIDATE cgroup2 *** [ 0.032271] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033158] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036029] Spectre V2 : User space: Vulnerable [ 0.037011] Speculative Store Bypass: Vulnerable [ 0.040244] debug: unmapping init [mem 0xffffffff8e459000-0xffffffff8e460fff] [ 0.042186] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043706] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044024] ... version: 2 [ 0.045015] ... bit width: 48 [ 0.046015] ... generic registers: 4 [ 0.047014] ... value mask: 0000ffffffffffff [ 0.048018] ... max period: 00007fffffffffff [ 0.049017] ... fixed-purpose events: 3 [ 0.050012] ... event mask: 000000070000000f [ 0.051274] rcu: Hierarchical SRCU implementation. [ 0.053374] smp: Bringing up secondary CPUs ... [ 0.054612] x86: Booting SMP configuration: [ 0.055033] .... node #0, CPUs: #1 #2 #3 [ 0.058513] smp: Brought up 1 node, 4 CPUs [ 0.060012] smpboot: Max logical packages: 1 [ 0.061019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.140016] node 0 deferred pages initialised in 76ms [ 0.144008] devtmpfs: initialized [ 0.145241] x86/mm: Memory block size: 128MB [ 0.147752] gcov: version magic: 0x41383552 [ 0.149344] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.153095] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156306] pinctrl core: initialized pinctrl subsystem [ 0.158169] [ 0.158844] ************************************************************* [ 0.162013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.165011] ** ** [ 0.167009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.170010] ** ** [ 0.172012] ** This means that this kernel is built to expose internal ** [ 0.175013] ** IOMMU data structures, which may compromise security on ** [ 0.178023] ** your system. ** [ 0.181013] ** ** [ 0.183008] ** If you see this message and you are not debugging the ** [ 0.184008] ** kernel, report this immediately to your vendor! ** [ 0.187013] ** ** [ 0.189011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.193012] ************************************************************* [ 0.195667] NET: Registered protocol family 16 [ 0.197415] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.200063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.203058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.206027] cpuidle: using governor menu [ 0.207721] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.209662] PCI: Using configuration type 1 for base access [ 0.212130] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.222121] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.225043] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.229042] cryptd: max_cpu_qlen set to 1000 [ 0.231414] ACPI: Added _OSI(Module Device) [ 0.233015] ACPI: Added _OSI(Processor Device) [ 0.235013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.237015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.242904] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.249477] ACPI: Interpreter enabled [ 0.251071] ACPI: PM: (supports S0 S3 S4 S5) [ 0.253017] ACPI: Using IOAPIC for interrupt routing [ 0.255110] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.259401] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.269541] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.272046] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.275022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.278076] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.284619] acpiphp: Slot [2] registered [ 0.286182] acpiphp: Slot [5] registered [ 0.288145] acpiphp: Slot [6] registered [ 0.290123] acpiphp: Slot [7] registered [ 0.291100] acpiphp: Slot [8] registered [ 0.293131] acpiphp: Slot [9] registered [ 0.295122] acpiphp: Slot [10] registered [ 0.296185] acpiphp: Slot [3] registered [ 0.298108] acpiphp: Slot [4] registered [ 0.300134] acpiphp: Slot [11] registered [ 0.302109] acpiphp: Slot [12] registered [ 0.303117] acpiphp: Slot [13] registered [ 0.305128] acpiphp: Slot [14] registered [ 0.307110] acpiphp: Slot [15] registered [ 0.309112] acpiphp: Slot [16] registered [ 0.310140] acpiphp: Slot [17] registered [ 0.312132] acpiphp: Slot [18] registered [ 0.314120] acpiphp: Slot [19] registered [ 0.316114] acpiphp: Slot [20] registered [ 0.317111] acpiphp: Slot [21] registered [ 0.319091] acpiphp: Slot [22] registered [ 0.321120] acpiphp: Slot [23] registered [ 0.322084] acpiphp: Slot [24] registered [ 0.324119] acpiphp: Slot [25] registered [ 0.325112] acpiphp: Slot [26] registered [ 0.327097] acpiphp: Slot [27] registered [ 0.328093] acpiphp: Slot [28] registered [ 0.329100] acpiphp: Slot [29] registered [ 0.331106] acpiphp: Slot [30] registered [ 0.333110] acpiphp: Slot [31] registered [ 0.334056] PCI host bridge to bus 0000:00 [ 0.336021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.338023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.341025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.344023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.347028] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.350053] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.352138] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.356046] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.359461] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.370015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.375958] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.379019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.381016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.384025] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.387593] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.390809] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.395049] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.398802] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.404014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.417881] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.422935] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.430350] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.437096] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.446016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.468017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.479828] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.487016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.495012] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.512017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.522662] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.529026] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.537022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.554016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.565260] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.574014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.580013] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.599014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.608651] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.617014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.623014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.639015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.647896] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.654012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.661022] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.674023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.685077] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.689412] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.692440] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.695428] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.698233] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.704125] iommu: Default domain type: Passthrough [ 0.705411] SCSI subsystem initialized [ 0.706390] ACPI: bus type USB registered [ 0.707176] usbcore: registered new interface driver usbfs [ 0.709172] usbcore: registered new interface driver hub [ 0.711191] usbcore: registered new device driver usb [ 0.713198] pps_core: LinuxPPS API ver. 1 registered [ 0.716022] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.719121] PTP clock support registered [ 0.721348] EDAC MC: Ver: 3.0.0 [ 0.723585] PCI: Using ACPI for IRQ routing [ 0.725867] NetLabel: Initializing [ 0.728012] NetLabel: domain hash size = 128 [ 0.729009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.731098] NetLabel: unlabeled traffic allowed by default [ 0.734170] vgaarb: loaded [ 0.736140] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.737023] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.745414] clocksource: Switched to clocksource kvm-clock [ 0.846328] VFS: Disk quotas dquot_6.6.0 [ 0.847793] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.850107] *** VALIDATE ramfs *** [ 0.851470] *** VALIDATE hugetlbfs *** [ 0.853074] pnp: PnP ACPI init [ 0.855649] pnp: PnP ACPI: found 6 devices [ 0.874521] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.877952] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.880539] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.882975] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.885431] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.887356] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.889589] NET: Registered protocol family 2 [ 0.891934] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.896921] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.901622] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.907605] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.911962] TCP: Hash tables configured (established 65536 bind 65536) [ 0.915594] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.919237] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.922621] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.926355] NET: Registered protocol family 1 [ 0.929319] RPC: Registered named UNIX socket transport module. [ 0.931633] RPC: Registered udp transport module. [ 0.933532] RPC: Registered tcp transport module. [ 0.935380] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.937779] NET: Registered protocol family 44 [ 0.939622] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.941927] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.944263] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.946381] PCI: CLS 0 bytes, default 64 [ 0.947830] Unpacking initramfs... [ 2.337731] debug: unmapping init [mem 0xffff91e8fcc54000-0xffff91e8fffbffff] [ 2.341866] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.344221] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.347495] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.836326] Initialise system trusted keyrings [ 2.839442] Key type blacklist registered [ 2.841732] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.852173] zbud: loaded [ 2.855906] *** VALIDATE nfs *** [ 2.857709] *** VALIDATE nfs4 *** [ 2.859400] pstore: using deflate compression [ 2.863189] Platform Keyring initialized [ 2.961544] NET: Registered protocol family 38 [ 2.963193] Key type asymmetric registered [ 2.964778] Asymmetric key parser 'x509' registered [ 2.967214] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.970769] io scheduler mq-deadline registered [ 2.972757] io scheduler kyber registered [ 2.974548] io scheduler bfq registered [ 2.976374] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.979140] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.982150] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.985090] ACPI: Power Button [PWRF] [ 2.990605] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.997395] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.016868] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.024448] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.041889] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.072546] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.103951] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.109564] Non-volatile memory driver v1.3 [ 3.111773] Linux agpgart interface v0.103 [ 3.145691] virtio_blk virtio1: [vda] 149784 512-byte logical blocks (76.7 MB/73.1 MiB) [ 3.149268] vda: detected capacity change from 0 to 76689408 [ 3.164358] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.168036] vdb: detected capacity change from 0 to 1073741824 [ 3.184490] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.188275] vdc: detected capacity change from 0 to 2621440000 [ 3.204969] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.208381] vdd: detected capacity change from 0 to 2621440000 [ 3.222507] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.225870] vde: detected capacity change from 0 to 4294967296 [ 3.241410] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.245053] vdf: detected capacity change from 0 to 4294967296 [ 3.251660] libphy: Fixed MDIO Bus: probed [ 3.256691] usbcore: registered new interface driver usbserial_generic [ 3.259465] usbserial: USB Serial support registered for generic [ 3.261571] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.266203] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.268489] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.271497] mousedev: PS/2 mouse device common for all mice [ 3.274627] rtc_cmos 00:05: RTC can wake from S4 [ 3.277730] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.279708] rtc_cmos 00:05: registered as rtc0 [ 3.284162] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.284670] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.287631] intel_pstate: CPU model not supported [ 3.292076] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.297777] hid: raw HID events driver (C) Jiri Kosina [ 3.300144] usbcore: registered new interface driver usbhid [ 3.302265] usbhid: USB HID core driver [ 3.303637] drop_monitor: Initializing network drop monitor service [ 3.306774] Initializing XFRM netlink socket [ 3.308847] NET: Registered protocol family 10 [ 3.311992] Segment Routing with IPv6 [ 3.313730] NET: Registered protocol family 17 [ 3.316117] mpls_gso: MPLS GSO support [ 3.322443] RAS: Correctable Errors collector initialized. [ 3.324809] AVX version of gcm_enc/dec engaged. [ 3.326781] AES CTR mode by8 optimization enabled [ 3.489332] sched_clock: Marking stable (3489311016, 0)->(4455146729, -965835713) [ 3.495700] registered taskstats version 1 [ 3.499700] Loading compiled-in X.509 certificates [ 3.504594] zswap: loaded using pool lzo/zbud [ 3.551320] Key type big_key registered [ 3.574372] Key type encrypted registered [ 3.579101] ima: No TPM chip found, activating TPM-bypass! [ 3.583825] ima: Allocated hash algorithm: sha1 [ 3.590739] ima: No architecture policies found [ 3.601468] evm: Initialising EVM extended attributes: [ 3.604805] evm: security.selinux [ 3.608343] evm: security.ima [ 3.609465] evm: security.capability [ 3.612182] evm: HMAC attrs: 0x1 [ 3.628984] rtc_cmos 00:05: setting system clock to 2026-09-10 12:15:15 UTC (1789042515) [ 3.639382] debug: unmapping init [mem 0xffffffff8f403000-0xffffffff8f5fffff] [ 3.643983] debug: unmapping init [mem 0xffffffff8e182000-0xffffffff8e458fff] [ 3.656066] Write protecting the kernel read-only data: 28672k [ 3.663641] debug: unmapping init [mem 0xffffffff8c803000-0xffffffff8c9fffff] [ 3.670651] debug: unmapping init [mem 0xffffffff8d114000-0xffffffff8d1fffff] [ 3.762452] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.786933] systemd[1]: Detected virtualization kvm. [ 3.796950] systemd[1]: Detected architecture x86-64. [ 3.803906] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.855754] systemd[1]: No hostname configured. [ 3.861950] systemd[1]: Set hostname to . [ 3.869874] random: systemd: uninitialized urandom read (16 bytes read) [ 3.874255] systemd[1]: Initializing machine ID from random generator. [ 3.993042] random: ln: uninitialized urandom read (6 bytes read) [ 4.156514] random: systemd: uninitialized urandom read (16 bytes read) [ 4.159733] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.169810] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.176176] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... Starting Setup Virtual Console... Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.314863] device-mapper: uevent: version 1.0.3 [ 5.318362] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 7.336385] virtio_net virtio0 ens2: renamed from eth0 [ 7.396817] random: fast init done Starting dracut initqueue hook... [ 7.557846] scsi host0: ata_piix [ 7.614666] scsi host1: ata_piix [ 7.630312] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.640192] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 13.137518] random: crng init done [ 13.143739] random: 7 urandom warning(s) missed due to ratelimiting [ 14.361045] dracut-initqueue[592]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 15.798181] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 19.421700] printk: systemd: 26 output lines suppressed due to ratelimiting [ 20.518650] SELinux: Disabled at runtime. [ 20.707367] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 20.760868] systemd[1]: Detected virtualization kvm. [ 20.764749] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 22.869331] systemd[1]: initrd-switch-root.service: Succeeded. [ 22.878196] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 22.886681] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 22.891833] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 22.896386] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 22.909940] systemd[1]: Starting Journal Service... Starting Journal Service... [ 22.920498] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. Starting Remount Root and Kernel File Systems... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-getty.slice. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. [ 23.379341] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 24.361796] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 25.555913] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 25.983218] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 26.670857] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 26.760477] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 32.151949] Key type dns_resolver registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ ***] A start job is running for Configur…only root support (10s / no limit) [ **] A start job is running for Configur…only root support (10s / no limit)[ 33.666832] NFS: Registering the id_resolver key type [ 33.676423] Key type id_resolver registered [ 33.687639] Key type id_legacy registered [ *] A start job is running for Configur…only root support (11s / no limit) [ **] A start job is running for Configur…only root support (11s / no limit) [ ***] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg350-server login: [ 94.813323] hrtimer: interrupt took 5019315 ns [ 106.168361] libcfs: loading out-of-tree module taints kernel. [ 106.215592] Key type ._llcrypt registered [ 106.217501] Key type .llcrypt registered [ 106.366242] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 128.134521] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 129.942474] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 129.957336] alg: No test for adler32 (adler32-zlib) [ 131.606560] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 132.817646] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 134.688152] Key type lgssc registered [ 136.431850] Lustre: Echo OBD driver; http://www.lustre.org/ [ 156.653047] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 208.362382] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 227.020707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 227.075287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 228.644020] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 228.697365] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 228.793850] Lustre: lustre-MDT0000: new disk, initializing [ 228.882685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 228.917631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 234.243314] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 249.148555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 249.240627] Lustre: 6519:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 249.266196] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 249.270119] Lustre: Skipped 1 previous similar message [ 249.331683] Lustre: lustre-MDT0001: new disk, initializing [ 249.407603] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 249.459064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 249.473726] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 254.406453] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 260.255815] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 272.433035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 272.732858] Lustre: lustre-OST0000: new disk, initializing [ 272.737513] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 272.750453] Lustre: 8459:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 272.842366] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 275.765273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 275.782429] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 275.890881] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 280.846969] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 299.336925] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 299.492748] Lustre: lustre-OST0001: new disk, initializing [ 299.500460] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 299.510173] Lustre: 9532:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 299.587704] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 307.055642] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 308.317856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 308.329354] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 308.421359] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 319.975532] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 328.835908] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 336.981481] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing check_logdir /tmp/testlogs/ [ 344.120583] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing yml_node [ 349.908849] Lustre: DEBUG MARKER: Client: 2.17.58.2 [ 353.156539] Lustre: DEBUG MARKER: MDS: 2.17.58.2 [ 356.873583] Lustre: DEBUG MARKER: OSS: 2.17.58.2 [ 359.644424] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 10 08:21:08 EDT 2026 [ 377.842624] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 379.468789] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 395.745624] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 395.749847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 395.763718] Lustre: Skipped 3 previous similar messages [ 395.775261] Lustre: Skipped 3 previous similar messages [ 400.870548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 400.872761] Lustre: Skipped 3 previous similar messages [ 401.788450] Lustre: server umount lustre-MDT0000 complete [ 406.272886] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789042918 with bad export cookie 15421421205544665749 [ 406.275974] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 406.297991] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 411.110573] LustreError: 6525:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 411.111247] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 411.113434] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 411.143336] LustreError: 6525:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 411.161329] Lustre: Skipped 1 previous similar message [ 412.763161] Lustre: server umount lustre-MDT0001 complete [ 416.291670] Lustre: server umount lustre-OST0000 complete [ 419.912828] Lustre: server umount lustre-OST0001 complete [ 425.834484] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 434.438286] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 469.370163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 469.542553] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 469.560525] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 469.618564] Lustre: lustre-MDT0000: new disk, initializing [ 469.692141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 469.713337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 474.168302] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 487.283655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 487.457120] Lustre: 15086:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 487.579650] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 487.585193] Lustre: Skipped 1 previous similar message [ 487.718432] Lustre: lustre-MDT0001: new disk, initializing [ 487.961117] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 488.045476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 488.063252] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 492.892482] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 502.520818] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 508.072869] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 514.658151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 514.993851] Lustre: lustre-OST0000: new disk, initializing [ 514.997793] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 515.004933] Lustre: 17051:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 515.091481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 516.938228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 516.950936] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 517.113875] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 521.208240] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 530.131165] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 534.836508] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 535.025362] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 539.449642] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 539.688694] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 542.691252] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 542.707238] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 542.725189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 542.731685] Lustre: Skipped 1 previous similar message [ 547.857933] Lustre: server umount lustre-OST0000 complete [ 552.932638] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 552.955367] Lustre: Skipped 1 previous similar message [ 552.966949] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 552.970230] Lustre: Skipped 2 previous similar messages [ 557.776544] Lustre: server umount lustre-MDT0000 complete [ 558.051305] LustreError: 15096:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 558.069219] LustreError: 15096:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 560.985516] LustreError: 15079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043072 with bad export cookie 15421421205544667282 [ 560.993144] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 560.999385] LustreError: 15079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 561.329850] Lustre: server umount lustre-MDT0001 complete [ 565.732737] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:24:35 (1789043075) [ 571.439415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 571.965741] LustreError: 19316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 572.019727] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 576.259164] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 577.505428] LustreError: 19317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 577.519860] LustreError: 19317:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 581.617152] LustreError: 19316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 585.033836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 589.788389] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 596.905684] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 602.152918] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 608.069670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 614.627810] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 622.424538] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 626.099899] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 631.777847] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 631.785490] LustreError: Skipped 1 previous similar message [ 631.792216] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 631.804970] Lustre: Skipped 1 previous similar message [ 631.810706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 637.860637] Lustre: server umount lustre-MDT0000 complete [ 641.084846] LustreError: 19297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043152 with bad export cookie 15421421205544668164 [ 641.092935] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 641.103082] LustreError: 19297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 641.284720] Lustre: server umount lustre-MDT0001 complete [ 655.181592] Lustre: server umount lustre-OST0000 complete [ 673.818177] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 683.267825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 683.645587] LustreError: 23699:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 683.727783] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 683.738995] Lustre: Skipped 2 previous similar messages [ 687.834064] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 694.946054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 699.648966] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 702.201935] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 707.929125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 714.061504] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 721.010336] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 721.126741] Lustre: lustre-OST0001: new disk, initializing [ 721.135263] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 721.144100] Lustre: 26014:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 722.841431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 722.867682] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 722.943474] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 727.370375] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 736.478916] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 739.637492] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 751.588945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 751.598530] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 751.618823] Lustre: Skipped 2 previous similar messages [ 751.631483] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 751.644341] Lustre: Skipped 6 previous similar messages [ 757.247441] Lustre: server umount lustre-MDT0000 complete [ 758.764220] LustreError: 24430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 758.793062] LustreError: 24430:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 761.354604] LustreError: 23681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043273 with bad export cookie 15421421205544669011 [ 761.357646] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 761.373105] LustreError: 23681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 761.773698] Lustre: server umount lustre-MDT0001 complete [ 775.902945] Lustre: server umount lustre-OST0000 complete [ 789.929749] Lustre: server umount lustre-OST0001 complete [ 809.264475] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 821.276744] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 821.691261] LustreError: 29035:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 821.761390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 821.769537] Lustre: Skipped 3 previous similar messages [ 827.692824] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 836.781929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 841.615377] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 844.601566] Lustre: 30183:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 851.808812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 858.133575] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 866.400647] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 872.970892] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 880.449088] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 883.992593] Lustre: 32059:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 892.392818] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 892.401403] Lustre: Skipped 5 previous similar messages [ 892.403620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 892.422996] Lustre: Skipped 7 previous similar messages [ 895.743279] Lustre: server umount lustre-MDT0000 complete [ 897.512539] LustreError: 29035:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 897.537326] LustreError: 29035:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 899.011481] LustreError: 29017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043410 with bad export cookie 15421421205544670306 [ 899.023593] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 899.332563] Lustre: server umount lustre-MDT0001 complete [ 912.354960] Lustre: server umount lustre-OST0000 complete [ 926.690155] Lustre: server umount lustre-OST0001 complete [ 936.867943] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:30:46 (1789043446) [ 968.892451] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 979.355551] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 984.629781] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 993.809914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 998.644618] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1001.737579] Lustre: 36017:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1008.898415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1016.258872] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1024.971607] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1031.923310] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1038.911642] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1042.106349] Lustre: 37895:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1050.593712] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1050.600158] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1050.617371] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1054.069450] Lustre: server umount lustre-MDT0000 complete [ 1056.225125] LustreError: 34867:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1056.237738] LustreError: 34867:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 1057.672311] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043569 with bad export cookie 15421421205544671412 [ 1057.688776] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1057.688892] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1057.974355] Lustre: server umount lustre-MDT0001 complete [ 1072.214904] Lustre: server umount lustre-OST0000 complete [ 1085.375654] Lustre: server umount lustre-OST0001 complete [ 1095.867208] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:33:25 (1789043605) [ 1113.641891] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1124.077501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1124.485219] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1124.492038] Lustre: Skipped 7 previous similar messages [ 1128.720979] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1136.770317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1141.105604] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1143.457500] Lustre: 41186:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1150.090733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1156.307406] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1163.696469] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1170.458765] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1178.401087] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1181.891750] Lustre: 43063:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1182.771433] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1182.781137] Lustre: Skipped 1 previous similar message [ 1185.765229] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1185.790738] Lustre: Skipped 4 previous similar messages [ 1185.794574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1185.800979] Lustre: Skipped 5 previous similar messages [ 1192.209177] Lustre: server umount lustre-MDT0000 complete [ 1195.689466] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043707 with bad export cookie 15421421205544672518 [ 1195.694040] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1195.705037] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1202.136645] Lustre: server umount lustre-MDT0001 complete [ 1211.392778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1220.579318] LustreError: 3648:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff91e944149f80 x1875947179868416/t0(0) o250->MGC192.168.203.150@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1225.672853] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1231.341571] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1235.779615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1241.600284] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1241.616275] Lustre: Skipped 1 previous similar message [ 1241.670296] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1250.814566] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1252.977678] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1260.032755] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1262.184065] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1272.301328] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1274.308932] Lustre: server umount lustre-MDT0000 complete [ 1278.240173] LustreError: 40016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043790 with bad export cookie 15421421205544673645 [ 1278.258524] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1278.260097] LustreError: 40016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1278.640345] Lustre: server umount lustre-MDT0001 complete [ 1292.946431] Lustre: server umount lustre-OST0000 complete [ 1307.158542] Lustre: server umount lustre-OST0001 complete [ 1316.094878] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 08:37:05 (1789043825) [ 1338.176785] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1349.629454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1350.222245] LustreError: 47701:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1350.249820] LustreError: 47701:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 25 previous similar messages [ 1355.963831] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1364.741897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1369.970917] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1372.650582] Lustre: 48853:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1378.900774] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1385.278529] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1393.772144] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1401.122436] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1409.282747] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1414.972517] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1432.034488] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1432.051895] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1432.059732] Lustre: Skipped 8 previous similar messages [ 1437.925486] Lustre: server umount lustre-MDT0000 complete [ 1441.404691] LustreError: 47681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789043953 with bad export cookie 15421421205544674436 [ 1441.406662] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1441.420185] LustreError: 47681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1441.793649] Lustre: server umount lustre-MDT0001 complete [ 1455.692092] Lustre: server umount lustre-OST0000 complete [ 1469.678687] Lustre: server umount lustre-OST0001 complete [ 1478.523446] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 08:39:48 (1789043988) [ 1480.964420] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1482.561961] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 08:39:52 (1789043992) [ 1495.240214] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 1505.126386] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1530.290528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1530.673341] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1530.790617] Lustre: lustre-MDT0000: new disk, initializing [ 1530.947105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1535.523902] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1544.731899] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1550.972334] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1551.242732] Lustre: lustre-OST0000: new disk, initializing [ 1551.252768] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1551.260523] Lustre: Skipped 1 previous similar message [ 1551.265904] Lustre: 54949:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1553.131530] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1553.140580] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1553.172738] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1557.424642] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1567.115333] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1569.473640] Lustre: DEBUG MARKER: create test files [ 1604.591450] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1604.597165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1604.624465] Lustre: Skipped 20 previous similar messages [ 1608.214597] Lustre: server umount lustre-OST0000 complete [ 1613.256762] Lustre: server umount lustre-MDT0000 complete [ 1627.615801] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 1630.927892] Key type lgssc unregistered [ 1631.278916] LNet: 56577:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1631.288658] LNetError: 56577:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1631.312318] LNet: Removed LNI 192.168.203.150@tcp [ 1632.416223] Key type .llcrypt unregistered [ 1632.418674] Key type ._llcrypt unregistered [ 1660.712091] Key type ._llcrypt registered [ 1660.719194] Key type .llcrypt registered [ 1660.887959] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1662.002497] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1662.139197] alg: No test for adler32 (adler32-zlib) [ 1663.250497] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 1663.602524] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 1665.384196] Key type lgssc registered [ 1666.668451] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1674.520370] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1674.545563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1676.146855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1680.717331] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1688.098669] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1694.915473] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1695.337866] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1700.471426] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1700.576937] Lustre: 59305:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1700.584825] Lustre: 59305:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1700.590758] Lustre: 59305:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1700.606469] Lustre: 59305:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1700.624119] Lustre: 59305:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1700.640704] Lustre: 59305:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1701.751598] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1709.565264] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1714.779282] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1721.305891] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1731.553875] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1731.568193] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1731.597626] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1735.851844] Lustre: server umount lustre-OST0000 complete [ 1740.661864] Lustre: server umount lustre-MDT0000 complete [ 1751.451750] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 1754.123591] Key type lgssc unregistered [ 1754.527845] LNet: 60916:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1754.532629] LNetError: 60916:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1754.562257] LNet: Removed LNI 192.168.203.150@tcp [ 1755.741851] Key type .llcrypt unregistered [ 1755.744343] Key type ._llcrypt unregistered [ 1787.342621] Key type ._llcrypt registered [ 1787.344261] Key type .llcrypt registered [ 1787.473593] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1789.141660] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1789.221923] alg: No test for adler32 (adler32-zlib) [ 1790.386522] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 1790.719514] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 1792.400126] Key type lgssc registered [ 1793.847648] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1802.441410] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1802.497842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1804.144274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1809.493645] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1817.057698] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1823.611423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1824.019654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1828.164953] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1828.271953] Lustre: 63749:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1828.280395] Lustre: 63749:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1828.287683] Lustre: 63749:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1828.298903] Lustre: 63749:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1828.316325] Lustre: 63749:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1828.329963] Lustre: 63749:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1829.688564] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1837.566667] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1843.322210] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1853.921495] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1853.931796] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1853.947770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1857.579847] Lustre: server umount lustre-OST0000 complete [ 1861.953358] Lustre: server umount lustre-MDT0000 complete [ 1872.490336] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 1875.304088] Key type lgssc unregistered [ 1875.591786] LNet: 65305:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1875.600247] LNetError: 65305:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1875.631127] LNet: Removed LNI 192.168.203.150@tcp [ 1876.448703] Key type .llcrypt unregistered [ 1876.451909] Key type ._llcrypt unregistered [ 1894.601992] Key type ._llcrypt registered [ 1894.604748] Key type .llcrypt registered [ 1894.691420] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 1912.178561] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1914.273218] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1914.440195] alg: No test for adler32 (adler32-zlib) [ 1915.796628] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 1916.389602] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 1918.328705] Key type lgssc registered [ 1920.745476] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1957.858993] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1957.872388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1959.095436] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1959.125973] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1959.261273] Lustre: lustre-MDT0000: new disk, initializing [ 1959.371791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1959.401888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1963.662210] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1973.939898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1974.078278] Lustre: 68454:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1974.185327] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1974.196111] Lustre: Skipped 1 previous similar message [ 1974.321269] Lustre: lustre-MDT0001: new disk, initializing [ 1974.458812] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1974.530636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1974.552036] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1979.464622] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1990.988185] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1997.698635] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2004.406368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2004.833953] Lustre: lustre-OST0000: new disk, initializing [ 2004.838131] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2004.843905] Lustre: 70426:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2004.921924] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2011.227870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2011.243479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2011.406908] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2011.841147] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2021.898640] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2028.340989] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2028.655397] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2034.702557] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2034.943369] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2042.337904] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2042.342169] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2042.351405] LustreError: Skipped 1 previous similar message [ 2042.374165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2042.398080] Lustre: Skipped 1 previous similar message [ 2042.437560] Lustre: Skipped 1 previous similar message [ 2044.019161] Lustre: server umount lustre-OST0000 complete [ 2051.554343] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2051.563314] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2051.593349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2052.598894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2052.608104] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2054.830060] Lustre: server umount lustre-MDT0000 complete [ 2058.864427] LustreError: 68449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789044570 with bad export cookie 9916628550160055685 [ 2058.871826] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2058.883939] LustreError: 68449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2059.177452] Lustre: server umount lustre-MDT0001 complete [ 2067.756310] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 08:49:37 (1789044577) [ 2080.677649] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 2130.437930] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 2139.430199] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 2181.794335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2181.980185] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2182.000579] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2182.077606] Lustre: lustre-MDT0000: new disk, initializing [ 2182.168308] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2182.205503] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2186.768698] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2198.932872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2199.097930] Lustre: 76450:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2199.180419] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2199.189910] Lustre: Skipped 1 previous similar message [ 2199.367140] Lustre: lustre-MDT0001: new disk, initializing [ 2199.560414] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2199.617840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2199.641232] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2205.681204] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2216.488720] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2223.733925] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2231.505429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2231.886566] Lustre: lustre-OST0000: new disk, initializing [ 2231.898449] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2231.910848] Lustre: 78419:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2232.060559] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2238.872677] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2240.578962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2240.594396] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2240.732652] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2248.736760] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2253.194951] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2253.416892] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2257.489193] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2257.673317] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2261.474790] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2261.479773] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2261.487668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2261.501014] Lustre: Skipped 1 previous similar message [ 2261.516752] Lustre: Skipped 2 previous similar messages [ 2265.503264] Lustre: server umount lustre-OST0000 complete [ 2271.205877] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2271.220059] LustreError: Skipped 1 previous similar message [ 2271.231715] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2271.247882] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2275.382369] Lustre: server umount lustre-MDT0000 complete [ 2276.836210] LustreError: 76455:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2276.881291] LustreError: 76455:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2279.282218] LustreError: 76442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789044791 with bad export cookie 9916628550160056567 [ 2279.283894] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2279.288517] LustreError: 76442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2279.776775] Lustre: server umount lustre-MDT0001 complete [ 2289.311354] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 08:53:18 (1789044798) [ 2296.890519] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2297.543255] LustreError: 80832:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2297.677947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2302.651904] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2302.948884] LustreError: 80833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2308.065743] LustreError: 80832:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2311.189181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2315.837307] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2322.721307] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2328.388428] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2334.454403] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2334.764477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2334.774492] Lustre: Skipped 1 previous similar message [ 2339.877917] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2347.299511] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2350.795498] Lustre: 83504:0:(genops.c:1782:obd_export_evict_by_uuid()) MGS: evicting 48c23d6b-14d1-4c5d-955d-d7d091929cb3 at adminstrative request [ 2353.121471] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2353.143049] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2363.361411] Lustre: MGS: Client 48c23d6b-14d1-4c5d-955d-d7d091929cb3 (at 0@lo) reconnecting [ 2363.387963] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x899ef12035c98469 to 0x899ef12035c98709 [ 2363.411971] Lustre: MGC192.168.203.150@tcp: Connection restored to 0@lo (at 0@lo) [ 2387.599200] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2387.826401] Lustre: lustre-OST0001: new disk, initializing [ 2387.832050] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2387.836348] Lustre: 83895:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2394.782541] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2395.771374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2395.791333] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2395.902833] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2404.436139] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2411.490138] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2411.495339] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2411.497050] LustreError: Skipped 1 previous similar message [ 2411.500784] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2411.500793] Lustre: Skipped 1 previous similar message [ 2411.536070] Lustre: Skipped 2 previous similar messages [ 2413.432902] Lustre: server umount lustre-OST0001 complete [ 2416.614328] LustreError: 83148:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2416.641063] LustreError: 83148:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2419.681726] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2419.699455] LustreError: Skipped 1 previous similar message [ 2423.860534] Lustre: server umount lustre-OST0000 complete [ 2429.409302] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2429.431534] Lustre: Skipped 2 previous similar messages [ 2429.444634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2429.448350] Lustre: Skipped 5 previous similar messages [ 2433.996625] Lustre: server umount lustre-MDT0000 complete [ 2437.090221] LustreError: 80828:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2437.107012] LustreError: 80828:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2437.736479] LustreError: 82675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789044949 with bad export cookie 9916628550160058121 [ 2437.746454] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2437.747033] LustreError: 82675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2438.212434] Lustre: server umount lustre-MDT0001 complete [ 2447.777531] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 08:55:57 (1789044957) [ 2449.350573] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2451.101915] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 08:56:01 (1789044961) [ 2452.678656] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2454.626886] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 08:56:04 (1789044964) [ 2456.307459] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2458.483139] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 08:56:08 (1789044968) [ 2460.466804] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2462.470573] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 08:56:12 (1789044972) [ 2464.463353] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2470.098148] loop: module loaded [ 2470.125517] loop0: detected capacity change from 0 to 4294967296 [ 2470.170578] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2471.964776] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2475.172984] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2479.168831] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2503.251134] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2503.278361] LustreError: 88020:0:(obd_config.c:920:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2503.287296] LustreError: 88020:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2503.294479] LustreError: 88020:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2503.307249] LustreError: 88020:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2521.126889] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 2530.247524] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 2566.037664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2566.330670] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2566.353042] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2566.457205] Lustre: lustre-MDT0000: new disk, initializing [ 2566.585622] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2566.598757] Lustre: Skipped 1 previous similar message [ 2566.619174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2571.097739] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2580.309711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2580.422768] Lustre: 90883:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2580.660700] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2584.919858] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2594.945797] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2600.931528] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2607.969806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2608.355321] Lustre: 92852:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2614.382551] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2614.577715] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2616.276404] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2626.378378] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2631.805330] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2632.038323] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2637.068330] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2637.321636] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2640.354084] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2640.361502] LustreError: Skipped 1 previous similar message [ 2640.362898] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2640.367315] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2640.379020] Lustre: Skipped 2 previous similar messages [ 2640.391513] Lustre: Skipped 3 previous similar messages [ 2645.474319] LustreError: 92838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2645.504529] LustreError: 92838:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2645.698331] Lustre: server umount lustre-OST0000 complete [ 2656.177407] Lustre: server umount lustre-MDT0000 complete [ 2660.334822] LustreError: 90877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789045172 with bad export cookie 9916628550160058737 [ 2660.348221] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2660.703841] Lustre: server umount lustre-MDT0001 complete [ 2668.850951] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 08:59:38 (1789045178) [ 2676.029515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2680.714775] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2681.830141] LustreError: 95265:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2681.849556] LustreError: 95265:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2690.200073] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2694.989748] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2702.739765] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2708.578259] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2715.536143] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2715.893506] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2715.900856] Lustre: Skipped 4 previous similar messages [ 2721.981250] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2728.919726] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2733.823028] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2734.066213] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2738.999553] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2739.223780] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2745.965402] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2746.151476] Lustre: lustre-OST0001: new disk, initializing [ 2746.154209] Lustre: Skipped 2 previous similar messages [ 2746.157621] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2746.160739] Lustre: Skipped 3 previous similar messages [ 2746.165205] Lustre: 98633:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2752.637278] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2754.724041] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2754.732480] Lustre: Skipped 2 previous similar messages [ 2754.735863] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2754.773211] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2762.193946] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2767.346840] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2767.742657] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2772.718316] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2773.017315] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2793.346188] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2816.423991] Lustre: Failing over lustre-MDT0000 [ 2816.494461] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2816.511774] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2816.512743] Lustre: Skipped 3 previous similar messages [ 2816.529834] Lustre: Skipped 6 previous similar messages [ 2816.869780] Lustre: server umount lustre-MDT0000 complete [ 2821.450071] LustreError: 95260:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2821.464697] LustreError: 95260:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2838.004745] Lustre: 66678:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789045333/real 1789045333] req@00000000563092e1 x1875949050409728/t0(0) o400->MGC192.168.203.150@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789045349 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2838.044822] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2844.595275] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2844.598477] LDISKFS-fs (dm-0): recovery complete [ 2844.622946] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2847.205155] LustreError: 66676:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000004b72b160 x1875949050417408/t0(0) o250->MGC192.168.203.150@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2847.666722] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2851.824178] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2852.661630] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2852.662035] Lustre: 95260:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2852.864856] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2852.934399] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2853.248120] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2853.266655] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2853.281763] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2853.614492] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2853.620370] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2853.936122] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2853.950165] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2854.288438] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2854.300262] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 2 previous similar messages [ 2854.624966] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2854.633794] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2854.961179] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2854.971262] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2856.563200] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2856.577651] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2856.914541] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2856.939304] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2857.256102] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2857.271667] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2860.760903] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2860.767064] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2861.090765] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2861.104562] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2861.425274] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2861.443514] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2869.067154] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2869.086815] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2869.096879] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2869.122063] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2869.456141] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2869.473215] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2885.305615] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2885.323693] Lustre: 101497:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2885.333071] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2885.341534] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2885.656195] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2885.663654] LustreError: 101497:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2912.273925] Lustre: 101497:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2912.287047] Lustre: 101497:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2912.301205] Lustre: 101497:0:(genops.c:1609:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client aff25690-c019-4342-ac75-d86186500f14@192.168.203.50@tcp [ 2912.330350] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2912.346221] Lustre: 101497:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000002616f911 x1875949029236480/t0(8589934783) o101->aff25690-c019-4342-ac75-d86186500f14@192.168.203.50@tcp:454/0 lens 592/0 e 0 to 0 dl 1789045434 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2912.399982] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2912.417061] Lustre: lustre-MDT0000: Denying connection for new client aff25690-c019-4342-ac75-d86186500f14 (at 192.168.203.50@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2912.464172] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2912.518193] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2912.522325] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2912.523483] Lustre: Skipped 2 previous similar messages [ 2912.584077] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 2912.585146] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2927.073813] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2940.384212] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2940.534415] Lustre: server umount lustre-OST0000 complete [ 2945.010185] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2945.023421] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2945.046311] Lustre: Skipped 5 previous similar messages [ 2945.049106] Lustre: Skipped 9 previous similar messages [ 2950.119162] LustreError: 98933:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2950.140838] LustreError: 98933:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 39 previous similar messages [ 2950.932591] Lustre: server umount lustre-OST0001 complete [ 2961.485938] Lustre: server umount lustre-MDT0000 complete [ 2965.992815] LustreError: 95247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789045477 with bad export cookie 9916628550160109949 [ 2966.003466] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2966.007474] LustreError: 95247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2966.466591] Lustre: server umount lustre-MDT0001 complete [ 2976.194342] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:04:45 (1789045485) [ 2985.052983] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2985.058814] Lustre: *** cfs_fail_loc=197, val=0*** [ 3000.736644] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3005.857891] LustreError: 103480:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.150@tcp: failed processing log, type 4: rc = -110 [ 3011.013665] Lustre: *** cfs_fail_loc=197, val=0*** [ 3011.019196] Lustre: Skipped 65 previous similar messages [ 3031.649605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3031.666443] Lustre: Skipped 2 previous similar messages [ 3037.633056] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3044.084924] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3045.940363] Lustre: server umount lustre-OST0000 complete [ 3054.511056] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:06:04 (1789045564) [ 3075.759576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3080.659381] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3090.052595] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3094.505960] LustreError: 105132:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3094.527619] LustreError: 105132:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3094.560475] LustreError: 105132:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 3095.155585] Lustre: server umount lustre-MDT0000 complete [ 3133.948391] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 3147.052399] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3184.509954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3184.801931] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3184.834280] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3184.923963] Lustre: lustre-MDT0000: new disk, initializing [ 3184.988270] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3189.950478] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3200.559448] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3200.647280] Lustre: 109307:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3200.939539] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3206.156212] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3216.575314] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3224.357401] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3233.049410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3233.513838] Lustre: 111273:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3241.714145] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3242.134932] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3253.026341] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3258.310268] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3258.663232] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3263.822852] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3264.227444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3267.553665] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3267.567628] LustreError: Skipped 1 previous similar message [ 3267.574684] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3267.609262] Lustre: Skipped 2 previous similar messages [ 3267.627065] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3267.641845] Lustre: Skipped 7 previous similar messages [ 3280.352290] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3280.547732] Lustre: server umount lustre-OST0000 complete [ 3291.142115] Lustre: server umount lustre-MDT0000 complete [ 3293.665561] LustreError: 109314:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3293.683723] LustreError: 109314:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 3296.042230] LustreError: 109301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789045807 with bad export cookie 9916628550160120673 [ 3296.054812] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3296.064958] LustreError: 109301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3296.420173] Lustre: server umount lustre-MDT0001 complete [ 3307.480653] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:10:17 (1789045817) [ 3317.864426] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 3321.466800] Key type lgssc unregistered [ 3321.782385] LNet: 113763:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3321.800943] LNetError: 113763:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3321.833930] LNet: Removed LNI 192.168.203.150@tcp [ 3322.823541] Key type .llcrypt unregistered [ 3322.828618] Key type ._llcrypt unregistered [ 3343.104148] Key type ._llcrypt registered [ 3343.106963] Key type .llcrypt registered [ 3343.250882] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 3361.183250] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3362.120818] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3362.338361] alg: No test for adler32 (adler32-zlib) [ 3363.467734] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 3363.668332] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 3365.377623] Key type lgssc registered [ 3366.515421] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3413.468191] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3413.527508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3414.821947] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3414.842961] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3414.913326] Lustre: lustre-MDT0000: new disk, initializing [ 3414.967480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3414.980990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3420.548117] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3432.350975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3432.775982] Lustre: lustre-OST0000: new disk, initializing [ 3432.782987] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3432.796177] Lustre: Skipped 1 previous similar message [ 3432.812784] Lustre: 117935:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3432.940624] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3439.791367] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3439.804282] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3439.848472] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3440.480418] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3451.088153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3451.283915] Lustre: lustre-OST0001: new disk, initializing [ 3451.287100] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3451.291823] Lustre: 118798:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3451.350330] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3458.817910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3458.841884] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3458.914057] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3459.313511] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3474.403238] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3474.434973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3474.450471] Lustre: Skipped 1 previous similar message [ 3475.928644] Lustre: server umount lustre-MDT0000 complete [ 3482.438089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3487.120375] LustreError: 117123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789045998 with bad export cookie 12091549862783926303 [ 3487.135494] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3487.140250] LustreError: 117123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3487.276973] Lustre: server umount lustre-OST0000 complete [ 3490.851631] Lustre: server umount lustre-OST0001 complete [ 3501.267912] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 09:13:30 (1789046010) [ 3524.892367] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 09:13:54 (1789046034) [ 3537.494512] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3561.293336] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:14:30 (1789046070) [ 3597.386580] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 3606.028554] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3639.325711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3639.584938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3639.613361] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3639.715462] Lustre: lustre-MDT0000: new disk, initializing [ 3639.787250] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3639.810536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3644.002173] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3653.215189] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3653.299873] Lustre: 126550:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3653.355297] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3653.360689] Lustre: Skipped 1 previous similar message [ 3653.449367] Lustre: lustre-MDT0001: new disk, initializing [ 3653.551882] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3653.589060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3653.597836] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3658.469592] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3668.344488] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3673.698719] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3680.559423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3680.834812] Lustre: lustre-OST0000: new disk, initializing [ 3680.838920] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3680.845991] Lustre: 128517:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3680.926349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3686.949943] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3686.958715] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3687.136947] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3687.782339] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3697.953622] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3703.512671] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3703.790557] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3708.892194] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3709.146633] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3712.999496] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3713.003660] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3713.007481] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3713.031506] Lustre: Skipped 2 previous similar messages [ 3717.533231] Lustre: server umount lustre-OST0000 complete [ 3723.235056] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3723.249810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3723.258948] Lustre: Skipped 1 previous similar message [ 3723.271967] Lustre: Skipped 2 previous similar messages [ 3728.359349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3728.370342] Lustre: Skipped 1 previous similar message [ 3732.454653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3732.463048] Lustre: Skipped 1 previous similar message [ 3736.032133] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3736.365951] Lustre: server umount lustre-MDT0000 complete [ 3740.209776] LustreError: 126541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789046252 with bad export cookie 12091549862783927535 [ 3740.217683] LustreError: 126541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3740.222985] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3740.550914] Lustre: server umount lustre-MDT0001 complete [ 3749.235153] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:17:39 (1789046259) [ 3755.449880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3756.008081] LustreError: 130930:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3756.028992] LustreError: 130930:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3756.075132] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3760.530714] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3761.122789] LustreError: 130931:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3765.217317] LustreError: 130930:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3769.317984] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3774.729572] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3783.937708] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3790.127310] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3796.954814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3797.374846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3797.389947] Lustre: Skipped 1 previous similar message [ 3804.246265] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3811.736987] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3819.475031] Lustre: *** cfs_fail_loc=159, val=0*** [ 3834.732131] Lustre: lustre-MDT0000: Client 72b27cdc-c6ef-424e-bd62-6b0340266326 (at 192.168.203.50@tcp) reconnecting [ 3836.343360] Lustre: *** cfs_fail_loc=159, val=0*** [ 3836.353179] Lustre: Skipped 5 previous similar messages [ 3852.645862] Lustre: lustre-MDT0000: Client 72b27cdc-c6ef-424e-bd62-6b0340266326 (at 192.168.203.50@tcp) reconnecting [ 3871.224581] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3871.225663] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3871.239506] LustreError: Skipped 1 previous similar message [ 3871.240279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3871.273619] Lustre: Skipped 1 previous similar message [ 3875.228547] Lustre: server umount lustre-OST0000 complete [ 3881.444262] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3885.044404] Lustre: server umount lustre-MDT0000 complete [ 3886.567813] LustreError: 133590:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3886.602440] LustreError: 133590:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3888.654263] LustreError: 130912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789046400 with bad export cookie 12091549862783928410 [ 3888.668580] LustreError: 130912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3888.668942] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3888.939718] Lustre: server umount lustre-MDT0001 complete [ 3899.081695] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 3901.722786] Key type lgssc unregistered [ 3902.033977] LNet: 134709:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3902.040510] LNetError: 134709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3902.059270] LNet: Removed LNI 192.168.203.150@tcp [ 3902.881583] Key type .llcrypt unregistered [ 3902.884178] Key type ._llcrypt unregistered [ 3924.512409] Key type ._llcrypt registered [ 3924.520547] Key type .llcrypt registered [ 3924.646750] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3925.383089] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3925.408107] alg: No test for adler32 (adler32-zlib) [ 3926.510964] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 3926.751806] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 3928.408273] Key type lgssc registered [ 3929.371637] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3936.766274] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3936.779051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3938.356868] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3942.827740] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3948.519902] LustreError: 136118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3948.559457] LustreError: 136118:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3952.209668] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3952.724109] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3957.344983] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3964.870842] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3970.205982] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3977.434055] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3977.976291] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3984.126648] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3984.254853] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3991.326876] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4002.275279] Lustre: *** cfs_fail_loc=159, val=0*** [ 4018.519496] Lustre: lustre-MDT0000: Client bf4a8701-bfcd-4156-8b8b-56d78d679f62 (at 192.168.203.50@tcp) reconnecting [ 4020.697490] Lustre: *** cfs_fail_loc=159, val=0*** [ 4036.967609] Lustre: lustre-MDT0001: Client bf4a8701-bfcd-4156-8b8b-56d78d679f62 (at 192.168.203.50@tcp) reconnecting [ 4038.532694] Lustre: *** cfs_fail_loc=159, val=0*** [ 4038.544476] Lustre: Skipped 3 previous similar messages [ 4053.886076] Lustre: lustre-MDT0001: Client bf4a8701-bfcd-4156-8b8b-56d78d679f62 (at 192.168.203.50@tcp) reconnecting [ 4059.290842] Lustre: 139154:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4062.566996] Lustre: *** cfs_fail_loc=159, val=0*** [ 4062.568871] Lustre: Skipped 4 previous similar messages [ 4077.926555] Lustre: lustre-MDT0000: Client ca1cf376-b15a-4ddc-819a-ad0b38bfb211 (at 192.168.203.50@tcp) reconnecting [ 4080.032726] Lustre: *** cfs_fail_loc=159, val=0*** [ 4080.043014] Lustre: Skipped 14 previous similar messages [ 4096.419240] Lustre: lustre-MDT0000: Client ca1cf376-b15a-4ddc-819a-ad0b38bfb211 (at 192.168.203.50@tcp) reconnecting [ 4098.692370] Lustre: 139447:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4117.476102] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4117.480055] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4117.492059] LustreError: Skipped 1 previous similar message [ 4117.495524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4117.526483] Lustre: Skipped 1 previous similar message [ 4122.597529] LustreError: 139462:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4122.763750] Lustre: server umount lustre-OST0000 complete [ 4126.689845] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4126.709615] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4126.732519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4126.744749] Lustre: Skipped 1 previous similar message [ 4127.714941] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4132.571946] Lustre: server umount lustre-MDT0000 complete [ 4132.835170] LustreError: 136118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4132.852808] LustreError: 136118:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4136.233669] LustreError: 139276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789046648 with bad export cookie 8202168942178872443 [ 4136.237653] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4136.247309] LustreError: 139276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4136.562720] Lustre: server umount lustre-MDT0001 complete [ 4146.348540] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4148.906546] Key type lgssc unregistered [ 4149.294764] LNet: 140523:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4149.301529] LNetError: 140523:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4149.324672] LNet: Removed LNI 192.168.203.150@tcp [ 4150.149502] Key type .llcrypt unregistered [ 4150.152935] Key type ._llcrypt unregistered [ 4173.424143] Key type ._llcrypt registered [ 4173.427064] Key type .llcrypt registered [ 4173.554786] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4174.781415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4174.838547] alg: No test for adler32 (adler32-zlib) [ 4175.909473] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 4176.134524] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4177.840167] Key type lgssc registered [ 4179.095821] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4186.750976] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4186.774414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4188.310177] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4192.677582] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4198.889159] LustreError: 141936:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4198.900408] LustreError: 141936:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4201.341767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4201.754865] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4206.076524] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4212.753541] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4217.707127] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4223.883059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4224.262660] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4226.349558] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4229.807268] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4233.537029] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4236.679301] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4242.845558] Lustre: 144513:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4245.854745] Lustre: 144562:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4247.253498] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4248.166821] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4251.719985] Lustre: 144708:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4252.440621] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4253.151604] Lustre: 144806:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4257.250528] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4257.265571] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4257.294351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4259.307064] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4259.308473] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4259.316584] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4259.334212] Lustre: Skipped 1 previous similar message [ 4263.263208] Lustre: server umount lustre-OST0000 complete [ 4268.514117] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4268.523516] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4268.538638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4274.658530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4274.672512] Lustre: Skipped 3 previous similar messages [ 4279.780042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4279.784937] Lustre: Skipped 1 previous similar message [ 4281.312281] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4281.461889] Lustre: server umount lustre-MDT0000 complete [ 4284.898087] LustreError: 141935:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4284.921032] LustreError: 141935:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4285.195301] LustreError: 141915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789046797 with bad export cookie 5987880276749845076 [ 4285.200898] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4285.204919] LustreError: 141915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4285.642819] Lustre: server umount lustre-MDT0001 complete [ 4297.504116] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4300.561331] Key type lgssc unregistered [ 4300.987662] LNet: 145867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4301.008197] LNetError: 145867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4301.042151] LNet: Removed LNI 192.168.203.150@tcp [ 4301.973791] Key type .llcrypt unregistered [ 4301.987447] Key type ._llcrypt unregistered [ 4325.826485] Key type ._llcrypt registered [ 4325.829684] Key type .llcrypt registered [ 4326.022359] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4326.980911] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4327.023742] alg: No test for adler32 (adler32-zlib) [ 4328.182716] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 4328.513359] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4330.208278] Key type lgssc registered [ 4331.383534] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4338.588481] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4338.616264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4340.022543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4344.133237] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4349.413717] LustreError: 147288:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4349.441568] LustreError: 147288:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4352.017433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4352.395144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4356.366745] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4363.110613] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4368.468732] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4374.276169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4374.599943] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4377.709699] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4379.957080] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4384.880552] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4386.622645] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4392.588974] Lustre: *** cfs_fail_loc=159, val=0*** [ 4409.142153] Lustre: lustre-MDT0000: Client 4b46ea08-3f49-437c-99f7-0825c919c960 (at 192.168.203.50@tcp) reconnecting [ 4415.977287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4415.990096] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4416.022658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4416.025018] Lustre: Skipped 1 previous similar message [ 4421.110889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4421.119093] Lustre: Skipped 1 previous similar message [ 4421.435445] Lustre: server umount lustre-OST0000 complete [ 4426.220154] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4426.226418] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4426.230750] Lustre: Skipped 2 previous similar messages [ 4426.242022] Lustre: Skipped 1 previous similar message [ 4430.938527] Lustre: server umount lustre-MDT0000 complete [ 4431.331858] LustreError: 147288:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4431.348859] LustreError: 147288:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4434.566643] LustreError: 147268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789046946 with bad export cookie 7113105243515963948 [ 4434.571078] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4434.580195] LustreError: 147268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4434.973957] Lustre: server umount lustre-MDT0001 complete [ 4444.808332] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4447.231803] Key type lgssc unregistered [ 4447.487447] LNet: 150958:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4447.496057] LNetError: 150958:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4447.521686] LNet: Removed LNI 192.168.203.150@tcp [ 4448.375248] Key type .llcrypt unregistered [ 4448.380620] Key type ._llcrypt unregistered [ 4475.428580] Key type ._llcrypt registered [ 4475.433548] Key type .llcrypt registered [ 4475.648935] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4477.277300] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4477.348439] alg: No test for adler32 (adler32-zlib) [ 4478.491705] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 4478.800869] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4480.464323] Key type lgssc registered [ 4482.054366] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4490.608936] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4490.643296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4492.025463] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4497.938367] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4501.478842] LustreError: 152368:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4501.505670] LustreError: 152368:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4506.597928] LustreError: 152367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4508.519743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4509.081487] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4513.879458] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4521.277648] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4527.562305] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4534.810777] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4535.281179] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4536.319061] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4541.170848] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4548.405103] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4553.719294] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4572.392911] Lustre: lustre-MDT0001: evicting 955130b0-946d-443b-ba3e-bdeb7094970b (at 192.168.203.50@tcp) by administrative request [ 4572.403099] Lustre: Skipped 1 previous similar message [ 4602.850111] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4602.863335] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4602.886942] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4604.902412] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4604.907411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4604.909880] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4604.952443] Lustre: Skipped 1 previous similar message [ 4606.789298] Lustre: server umount lustre-OST0000 complete [ 4611.553734] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4611.562410] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4611.594193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4615.139518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4615.140980] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4617.576660] Lustre: server umount lustre-MDT0000 complete [ 4620.259297] LustreError: 152367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4620.271244] LustreError: 152367:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4621.881054] LustreError: 152349:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789047133 with bad export cookie 5135861745442176758 [ 4621.922194] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4622.568581] Lustre: server umount lustre-MDT0001 complete [ 4633.790124] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4636.437262] Key type lgssc unregistered [ 4636.741808] LNet: 156431:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4636.750796] LNetError: 156431:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4636.784036] LNet: Removed LNI 192.168.203.150@tcp [ 4637.630926] Key type .llcrypt unregistered [ 4637.634882] Key type ._llcrypt unregistered [ 4710.541283] Key type ._llcrypt registered [ 4710.545655] Key type .llcrypt registered [ 4710.665797] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4711.653412] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4711.810605] alg: No test for adler32 (adler32-zlib) [ 4712.897282] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 4713.164879] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4714.864166] Key type lgssc registered [ 4716.129518] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4723.354194] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4723.370761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4724.825908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4729.233364] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4734.946939] LustreError: 158917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4734.956343] LustreError: 158917:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4737.280952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4737.738790] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4742.153914] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4748.942594] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4754.764984] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4761.932591] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4762.549245] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4763.576300] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4768.513118] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4769.727940] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4776.873456] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4789.218926] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4789.229255] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4789.243342] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4790.247637] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4790.249801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4790.260435] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4793.121601] Lustre: server umount lustre-OST0000 complete [ 4798.952755] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4798.959231] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4798.973410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4798.978217] Lustre: Skipped 1 previous similar message [ 4804.118499] Lustre: server umount lustre-MDT0000 complete [ 4805.601438] LustreError: 158917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4805.627300] LustreError: 158917:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4807.794320] LustreError: 158898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789047319 with bad export cookie 15049370505506304207 [ 4807.797672] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4807.812463] LustreError: 158898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4808.211136] Lustre: server umount lustre-MDT0001 complete [ 4819.124524] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4822.302734] Key type lgssc unregistered [ 4822.648323] LNet: 162461:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4822.668563] LNetError: 162461:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4822.688649] LNet: Removed LNI 192.168.203.150@tcp [ 4823.578968] Key type .llcrypt unregistered [ 4823.581529] Key type ._llcrypt unregistered [ 4850.420952] Key type ._llcrypt registered [ 4850.425852] Key type .llcrypt registered [ 4850.594647] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4852.300507] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4852.350315] alg: No test for adler32 (adler32-zlib) [ 4853.486567] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 4853.753344] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4855.432177] Key type lgssc registered [ 4856.511027] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4871.456969] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 09:36:21 (1789047381) [ 4888.029290] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 4899.037753] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4936.394058] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4936.411905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4937.751576] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4937.786325] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4937.847941] Lustre: lustre-MDT0000: new disk, initializing [ 4937.986357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4938.010106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4942.815271] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4953.523915] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4953.669894] Lustre: 166724:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4953.751341] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4953.755852] Lustre: Skipped 1 previous similar message [ 4953.840412] Lustre: lustre-MDT0001: new disk, initializing [ 4954.007818] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4954.091457] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4954.113048] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4958.840748] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4969.794256] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4977.137436] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4984.909508] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4985.229092] Lustre: lustre-OST0000: new disk, initializing [ 4985.233614] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4985.240751] Lustre: 168692:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4985.361166] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4992.329591] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4993.596486] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4993.609278] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4993.750714] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5002.709066] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5014.504570] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5014.504617] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5014.519199] LustreError: Skipped 1 previous similar message [ 5014.522960] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5014.553805] Lustre: Skipped 1 previous similar message [ 5019.555050] Lustre: server umount lustre-OST0000 complete [ 5024.741225] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5024.754213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5024.764527] Lustre: Skipped 2 previous similar messages [ 5029.773505] Lustre: server umount lustre-MDT0000 complete [ 5033.612771] LustreError: 166714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789047545 with bad export cookie 2640910715972062433 [ 5033.616171] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5033.624609] LustreError: 166714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5034.122928] Lustre: server umount lustre-MDT0001 complete [ 5044.847418] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 5047.530631] Key type lgssc unregistered [ 5047.808154] LNet: 170598:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5047.815130] LNetError: 170598:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5047.828239] LNet: Removed LNI 192.168.203.150@tcp [ 5048.548046] Key type .llcrypt unregistered [ 5048.550862] Key type ._llcrypt unregistered [ 5073.159773] Key type ._llcrypt registered [ 5073.163179] Key type .llcrypt registered [ 5073.258700] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 5074.117742] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5074.201040] alg: No test for adler32 (adler32-zlib) [ 5075.224369] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 5075.499416] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 5077.240173] Key type lgssc registered [ 5078.350856] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5085.765671] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5085.785917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5087.288587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5091.985561] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 5097.448312] LustreError: 172010:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5097.486791] LustreError: 172010:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5100.873191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5101.220063] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5105.114907] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 5111.314788] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5116.117938] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5122.827039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5123.189203] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5128.747553] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 5132.413568] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5136.718740] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5142.861229] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5142.877665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5142.898066] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5142.910669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.931012] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5143.860517] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5143.877072] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5143.910696] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5143.922835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.938763] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5144.987729] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5145.005862] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5145.014728] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5145.019784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.029385] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5145.960982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.870369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.685392] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5147.695463] Lustre: Skipped 2 previous similar messages [ 5147.712127] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5147.724547] Lustre: Skipped 2 previous similar messages [ 5147.731391] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5147.742587] Lustre: Skipped 2 previous similar messages [ 5147.744983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.759335] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5147.765570] Lustre: Skipped 2 previous similar messages [ 5148.821466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.877797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.839658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.770284] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5151.775470] Lustre: Skipped 3 previous similar messages [ 5151.799827] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5151.819339] Lustre: Skipped 3 previous similar messages [ 5151.823837] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5151.829249] Lustre: Skipped 3 previous similar messages [ 5151.836439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.851758] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5151.854704] Lustre: Skipped 3 previous similar messages [ 5153.009766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.834657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.692858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.540450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.440862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.413281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.515793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.537959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.321073] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5160.327194] Lustre: Skipped 8 previous similar messages [ 5160.355830] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5160.381456] Lustre: Skipped 8 previous similar messages [ 5160.389778] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5160.393055] Lustre: Skipped 8 previous similar messages [ 5160.398782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.412610] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5160.422475] Lustre: Skipped 8 previous similar messages [ 5161.208365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.079294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.863424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.897635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.752388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.843484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.625851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.580060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.539806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.404400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.169603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.955971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.800470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.725627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.561945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.398790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.131033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.927563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.663163] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5176.676934] Lustre: Skipped 18 previous similar messages [ 5176.704283] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5176.714772] Lustre: Skipped 18 previous similar messages [ 5176.719615] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5176.724651] Lustre: Skipped 18 previous similar messages [ 5176.728773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.743171] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5176.752303] Lustre: Skipped 18 previous similar messages [ 5177.700122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.539070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.583329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.533677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.514542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.442458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.551234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.506982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.374218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.276241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.276675] LustreError: 172029:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5187.287493] LustreError: 172029:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5187.294221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.104530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.970262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.985818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.931715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.765958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.618933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.456449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.318977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.116209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.025238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.854633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.730944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.521591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.382507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.178678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.036946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.980693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.797919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.657650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.583162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.386601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.134829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.966597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.692232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.528494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.338731] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5209.347770] Lustre: Skipped 36 previous similar messages [ 5209.365556] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5209.381640] Lustre: Skipped 36 previous similar messages [ 5209.387857] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5209.391702] Lustre: Skipped 36 previous similar messages [ 5209.396514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.410259] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5209.417873] Lustre: Skipped 36 previous similar messages [ 5210.229381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.056922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.959504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.728612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.547289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.624534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.446968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.313224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.218785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.130542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.016345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.026653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.871536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.744647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.444625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.462570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.390600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.347693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.226557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.006272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.793551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.671546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.497371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.547426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.268604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.987278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.776258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.619463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.538566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.484976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.351566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.119760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.944234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.805264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.715525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.700490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.595179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.502812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.393073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.296726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.143205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.152405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.075210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.976877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.834904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.781570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.557279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.416440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.142757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.960222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.708340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.521643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.267976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.148734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.047714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.861577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.855879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.673865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.419264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.363294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.242478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.227228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.215731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.103171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.197274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.830833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.762405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.730725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.881817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.240799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.422426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.547762] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5273.560333] Lustre: Skipped 71 previous similar messages [ 5273.593882] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5273.605757] Lustre: Skipped 71 previous similar messages [ 5273.610881] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5273.617229] Lustre: Skipped 71 previous similar messages [ 5273.621485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.633409] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5273.639944] Lustre: Skipped 71 previous similar messages [ 5274.550663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.508882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.675652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.203621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.677913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.157728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.281552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.201648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.665365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.936387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.347970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.091739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.573602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.353878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.581787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.706895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.101611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.347570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.289163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.860064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.892063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.764891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.712246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.957144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.238536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.406276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.537093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.510460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.663764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.753734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.116231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.057286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.267408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.488534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.349570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.381225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.329876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.353188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.201353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.331517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.107826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.804751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.750023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.669686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.478034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.269492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.030969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.774736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.780929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.858921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.899886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.008869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.968553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.044934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.399762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.459142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.506976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.963643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.064301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.157629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.269929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.517814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.722497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.819073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.970423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.012630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.806161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.756969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.725482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.762888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.708553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.483069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.404140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.243826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.224466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.177690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.164349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.042954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.307566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.264644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.081758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.932225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.810743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.077348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.131491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.123927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.346949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.331382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.456536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.382686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.395718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.302375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.335287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.378530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.579811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.475115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.405169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.360978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.287629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.300698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.233772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.257894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.191232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.073927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.025846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.776607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.666712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.587779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.444860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.195807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.151470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.979304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.890266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.673751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.681862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.547411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.377962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.307711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.234431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.229508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.171788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.160375] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5402.165565] Lustre: Skipped 121 previous similar messages [ 5402.190936] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5402.203491] Lustre: Skipped 121 previous similar messages [ 5402.209248] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5402.213966] Lustre: Skipped 121 previous similar messages [ 5402.220403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.231538] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5402.237883] Lustre: Skipped 121 previous similar messages [ 5403.241196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.214534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.100200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.198943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.081935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.899588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.822432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.874543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.739178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.754925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.701332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.742296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.630298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.479351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.346706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.210513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.077837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.104579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.115137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.044373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.003975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.048686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.928775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.897565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.887544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.637924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.637394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.689488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.929383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.126713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.231803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.307450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.330479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.324534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.434488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.457888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.401805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.213271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.048902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.775363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.767447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.627505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.637602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.506515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.327798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.095923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.875871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.491781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.398829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.173508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.135644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.111581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.933579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.834387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.636599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.393209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.328510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.107535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.938603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.829736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.827684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.633106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.597620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.546063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.392514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.098606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.048505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.824981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.806548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.760659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.858398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.945459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.899404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.986723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.024703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.182520] LustreError: 172029:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5473.194745] LustreError: 172029:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5473.200857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.108916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.087597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.964985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.848289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.851472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.784628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.828976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.680590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.731703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.540577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.477151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.457896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.606913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.721945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.673508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.699733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.615255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.605310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.448676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.459156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.275923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.156263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.213857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.038882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.238681] LustreError: 172029:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5497.258821] LustreError: 172029:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5497.263513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.299070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.322273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.534503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.641280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.600216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.510241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.380272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.152107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.049374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.955073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.074276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.921522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.920070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.640816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.675091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.613568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.712332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.585615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.659029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.748517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.807154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.722665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.748294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.885530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.037212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.109362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.156327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.435338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.566429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.523719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.618254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.957853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.002705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.983090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.031282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.988389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.684967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.381228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.350757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.276356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.314040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.237948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.326127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.444723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.559488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.315631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.149425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.087397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.178209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.995377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.928915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.817495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.770073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.531442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.438380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.384143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.269280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.124277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.891247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.730328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.713877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.498734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.320108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.097157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.916318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.592587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.301229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.929660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.614265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.409245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.388372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.298441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.102399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.207732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.131335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.124985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.024554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.914357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.889389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.719638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.671070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.663376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.451339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.359778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.145697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.979730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.879616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.731170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.509098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.333380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.195334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.969481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.665448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.428471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.251960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.019817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.917610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.607549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.516808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.392400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.341704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.113239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.071681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.838502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.664067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.490243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.292578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.966611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.685848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.402702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.388510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.213584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.040765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.780855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.560539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.396497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.247516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.100147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.866603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.571162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.375690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.077864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.788392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.589436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.388470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.049682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.972597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.780583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.699828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.478589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.449418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.270741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.147321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.966266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.810161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.541481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.382112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.223638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.042369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.806505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.683334] LustreError: 172029:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5623.706594] LustreError: 172029:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5623.707686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.581711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.459773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.166624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.974809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.865268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.702213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.585624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.458832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.260546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.145525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.027660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.903777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.742873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.540571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.333664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.083947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.707289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.535355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.281337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.245970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.157062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.995694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.796734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.691227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.494601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.287421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.932379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.792301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.575520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.477094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.253840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.993782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.771583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.485167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.260655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.997384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.769574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.578435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.324108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.138815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.890792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.693327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.347481] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5658.352429] Lustre: Skipped 284 previous similar messages [ 5658.366957] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5658.377167] Lustre: Skipped 284 previous similar messages [ 5658.381696] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5658.386702] Lustre: Skipped 284 previous similar messages [ 5658.391888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.404983] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5658.422445] Lustre: Skipped 284 previous similar messages [ 5659.215496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.917702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.698495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.579491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.541618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.311270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.158621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.051810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.955582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.705699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.603899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.324785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.233832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.018337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.851230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.578354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.488713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.359492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.330881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.111049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.879416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.625498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.513157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.315437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.193248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.112611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.395106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.372992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.311070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.287766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.386279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.264563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.214718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.091521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.022818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.831823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.636802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.455516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.253383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.094226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.975979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.833650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.974917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.873319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.904606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.952738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.348275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.482273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.379801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.188527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.932310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.755418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.726134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.979497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.152639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.473285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.678211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.824339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.242746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.195991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.683760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.972043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.293575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.480529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.875863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.086776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.252269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.244143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.712721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.974826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.998546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.277443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.298724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.609273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.383540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.548019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.904568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.917556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.029057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.311553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.861584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.976570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.672994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.480732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.858381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.661519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.506293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.267635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.028582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.865277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.636996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.368338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.225784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.035690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.834548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.618055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.625758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.435247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.345317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.146424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.942270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.814709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.596292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.446420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.248398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.992098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.839874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.704505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.525134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.423076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.186983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.921280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.702679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.526865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.286241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.054466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.876843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.639347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.408590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.219377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.080558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.865992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.707770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.684463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.634207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.543717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.352673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.130180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.039723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.025753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.819361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.734096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.554085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.543646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.319124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.304747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.162184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.007190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.841861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.714781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.567585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.516128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.449248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.299344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.194785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.087818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.005056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.855410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.790422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.783746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.609423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.526405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.488373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.500366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.546144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.551770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.441157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.525123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.676945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.658214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.698577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.581344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.591346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.458793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.407196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.530077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.484638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.482317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.296807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.240429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.097641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.190501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.278413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.496254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.587057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.693717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.877677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.835696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.884783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.980354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.081979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.165659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.275894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.210217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.411700] LustreError: 172029:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5836.425687] LustreError: 172029:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5836.441190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.331817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.410693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.465994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.648476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.599934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.376359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.223647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.978985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.758762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.422391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.305790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.178343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.074283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.893701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.768892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.674966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.535780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.373667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.447901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.493554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.386098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.586269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.484322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.391072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.341237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.278823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.170906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.036294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.029534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.060122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.935972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.913078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.720302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.562866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.315384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.066470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.820742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.707675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.620266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.594430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.798297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.698454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.421423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.091740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.859358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.702328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.537016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.240261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.167227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.036253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.900871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.736362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.655549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.594897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.565659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.445755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.370219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.363427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.354807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.242387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.191836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.113169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.943516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.895880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.746090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.665771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.610908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.599094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.844950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.787712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.785551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.635215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.583995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.508506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.425528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.183486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.109979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.003136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.941303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.804704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.768444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.791725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.737137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.627946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.620551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.502909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.555309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.571888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.250322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.326706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.325246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.345576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.488320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.440654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.621259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.527057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.483526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.685714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.689584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.548800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.501056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.348840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.259353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.231744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.104982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.173824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.091847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.020942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.891904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.774664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.750600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.614043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.482391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.323609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.191896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.100361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.999864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.026618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.888006] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.672526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.532203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.305831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.287547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.130460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.187918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.509383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.633903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.711315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.577103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.511661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.552431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.618226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.537889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.654589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.427722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.399983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.394426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.389065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.321878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.127650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.071673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.011720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.157959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.084134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.046554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.949509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.857809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.108211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.986248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.112811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.031950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.162364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.136394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.090691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.216603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.072639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.019128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.916125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.852926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.856721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.069420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.960190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.005770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.833969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.863418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.938242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.959878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.827279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.705488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.687840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.494149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.524098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.461568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.771356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.824588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.744047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.556465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.526943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.465567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.460262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.359689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.369081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.346138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.164982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.009614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.850406] LustreError: 172029:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6010.865370] LustreError: 172029:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6010.873053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.692554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.926817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.969419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.138516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.036305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.086272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.008544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.005993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.928090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.847482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.758822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.758703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.625341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.602883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.423152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.374708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.232952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.194724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.984575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.950489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.798312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.711975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.637528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.631392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.446899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.440256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.399056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.378911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.344835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.400083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.174156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.960984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.743761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.642863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.412378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.339374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.229597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.119209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.144587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.130534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.113817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.040158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.875207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.020592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.274289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.386252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.343077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.204883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.133809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.156454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.079613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.145194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.058614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.968976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.853027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.865329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.689255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.701368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.617772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.551748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.458760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.481194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.710811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.556821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.525442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.495148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.742390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.645636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.735165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.723806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.791634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.680243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.854356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.650123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.585570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.026655] LustreError: 173840:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6453.049659] LustreError: 172029:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7009.771961] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7009.775443] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7009.782055] LustreError: Skipped 1 previous similar message [ 7009.802453] Lustre: Skipped 446 previous similar messages [ 7009.816432] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7010.792797] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7010.795122] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7010.802905] Lustre: Skipped 1 previous similar message [ 7010.829474] LustreError: Skipped 6 previous similar messages [ 7012.239948] Lustre: server umount lustre-OST0000 complete [ 7021.027042] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7021.031939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7021.035183] Lustre: Skipped 1 previous similar message [ 7022.640974] Lustre: server umount lustre-MDT0000 complete [ 7025.560499] LustreError: 171990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789049537 with bad export cookie 14935308644783326683 [ 7025.563952] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7025.573175] LustreError: 171990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7025.814876] Lustre: server umount lustre-MDT0001 complete [ 7033.606828] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 7035.890279] Key type lgssc unregistered [ 7036.114583] LNet: 227261:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7036.126313] LNetError: 227261:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7036.147618] LNet: Removed LNI 192.168.203.150@tcp [ 7036.861563] Key type .llcrypt unregistered [ 7036.868905] Key type ._llcrypt unregistered [ 7057.281908] Key type ._llcrypt registered [ 7057.285206] Key type .llcrypt registered [ 7057.374685] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7058.502893] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7058.540290] alg: No test for adler32 (adler32-zlib) [ 7059.589530] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 7059.855613] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 7061.544151] Key type lgssc registered [ 7062.558104] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7070.313497] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7070.332202] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7071.656586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7075.646893] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7081.960204] LustreError: 228652:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7081.979960] LustreError: 228652:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7082.503480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7082.787708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7086.128815] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7092.230727] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7096.829937] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7102.301374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7102.562154] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7108.359538] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7115.419387] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7116.927767] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7121.014872] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7122.407660] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7122.418127] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7122.428672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7122.445438] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7122.445671] LustreError: 228671:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7127.540992] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7127.558477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7127.561771] Lustre: Skipped 1 previous similar message [ 7127.572060] Lustre: Skipped 1 previous similar message [ 7128.915923] Lustre: lustre-OST0000: Not available for connect from 192.168.203.50@tcp (stopping) [ 7130.295712] Lustre: server umount lustre-OST0000 complete [ 7142.472133] LustreError: 228671:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7163.852460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7164.338736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7167.408297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.427476] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.436569] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7167.445352] Lustre: Skipped 1 previous similar message [ 7170.850565] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7177.943937] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7179.294725] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7184.723984] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa072078d7000.ost_server_uuid 50 [ 7186.255880] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa072078d7000.ost_server_uuid in FULL state after 0 sec [ 7191.523523] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7191.536763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7191.554427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7192.500327] Lustre: server umount lustre-OST0000 complete [ 7195.620292] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7195.629093] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7195.647554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7201.886890] Lustre: server umount lustre-MDT0000 complete [ 7203.299854] LustreError: 229747:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7203.326072] LustreError: 229747:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7205.255540] LustreError: 228632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789049717 with bad export cookie 4628825407839979983 [ 7205.268616] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7205.276051] LustreError: 228632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7215.745859] Lustre: server umount lustre-MDT0001 complete [ 7224.561257] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 7226.780865] Key type lgssc unregistered [ 7227.003665] LNet: 233855:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7227.010701] LNetError: 233855:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7227.031835] LNet: Removed LNI 192.168.203.150@tcp [ 7227.635235] Key type .llcrypt unregistered [ 7227.637081] Key type ._llcrypt unregistered [ 7253.365486] Key type ._llcrypt registered [ 7253.368199] Key type .llcrypt registered [ 7253.441557] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7254.386510] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7254.579161] alg: No test for adler32 (adler32-zlib) [ 7255.614256] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 7256.014256] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 7257.768177] Key type lgssc registered [ 7259.042729] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7267.608610] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7267.633875] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7269.151970] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7274.693001] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7278.563242] LustreError: 235572:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7278.591735] LustreError: 235572:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7283.682143] LustreError: 235573:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7284.462743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7284.829618] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7288.934555] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7291.635544] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7298.447640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7298.994879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7305.425771] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7313.346969] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7314.948851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7315.198933] Lustre: lustre-OST0001: new disk, initializing [ 7315.208915] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7315.228291] Lustre: 237884:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7315.358518] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7320.710389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7320.729743] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7320.881821] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7322.145311] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7334.162945] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7339.982632] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7351.783094] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7351.788439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7351.825923] Lustre: Skipped 2 previous similar messages [ 7352.628364] Lustre: server umount lustre-MDT0000 complete [ 7352.819038] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7352.834740] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7352.853804] LustreError: 236301:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7356.900066] LustreError: 235568:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7356.925038] LustreError: 235568:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7357.046417] LustreError: 235552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789049868 with bad export cookie 4316994926336062613 [ 7357.056315] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7357.064680] LustreError: 235552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7357.911680] Lustre: server umount lustre-MDT0001 complete [ 7371.404518] Lustre: server umount lustre-OST0000 complete [ 7386.589588] Lustre: server umount lustre-OST0001 complete [ 7407.132704] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7416.479511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7416.876849] LustreError: 240993:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7416.950439] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7420.703804] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7426.530063] LustreError: 240994:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7426.562038] LustreError: 240994:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7428.871434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7429.220329] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7433.512753] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7436.107376] Lustre: 242135:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7442.949148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7448.736469] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7454.578402] LustreError: 242490:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7454.594585] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7456.484284] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7456.678703] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7456.690428] Lustre: Skipped 1 previous similar message [ 7461.558767] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7461.831106] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7461.832627] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7467.628232] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7470.417578] Lustre: 244006:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7480.291888] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7480.299720] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7480.310314] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7480.319510] Lustre: Skipped 2 previous similar messages [ 7482.343035] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7482.343363] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7482.363678] Lustre: Skipped 2 previous similar messages [ 7482.380287] Lustre: Skipped 3 previous similar messages [ 7485.404851] Lustre: server umount mylustre-MDT0000 complete [ 7487.460143] LustreError: 241727:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7487.496178] LustreError: 241727:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7489.375818] LustreError: 240970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789050001 with bad export cookie 4316994926336065476 [ 7489.378182] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7489.392483] LustreError: 240970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7489.952626] Lustre: server umount mylustre-MDT0001 complete [ 7503.255438] Lustre: server umount mylustre-OST0000 complete [ 7516.941899] Lustre: server umount mylustre-OST0001 complete [ 7542.534961] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7553.227457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7553.497532] LustreError: 246317:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7553.569613] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7558.566820] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7567.695781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7572.309962] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7574.682749] Lustre: 247459:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7580.321404] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7586.270549] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7592.909711] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 7594.059967] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7598.424546] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7598.434251] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7600.327522] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7606.980428] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7609.976679] Lustre: 249330:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7619.044093] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7619.054889] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7619.062159] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7619.069473] Lustre: Skipped 3 previous similar messages [ 7619.072098] Lustre: Skipped 2 previous similar messages [ 7622.578139] Lustre: server umount tfs-MDT0000 complete [ 7626.260132] LustreError: 246294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789050138 with bad export cookie 4316994926336067653 [ 7626.274478] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7626.277457] LustreError: 246294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7626.632291] Lustre: server umount tfs-MDT0001 complete [ 7640.506204] Lustre: server umount tfs-OST0000 complete [ 7653.309189] Lustre: server umount tfs-OST0001 complete [ 7674.165319] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7685.789036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7686.266596] LustreError: 251586:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7686.282095] LustreError: 251586:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7686.345209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7686.348130] Lustre: Skipped 3 previous similar messages [ 7690.812045] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7698.438854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7702.390876] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7704.833841] Lustre: 252727:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7710.901533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7712.178274] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7716.993615] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7723.511128] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 7726.360479] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7731.703805] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7733.359781] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7740.826964] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7744.518703] Lustre: 254597:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7751.644373] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:24:21 (1789050261) [ 7770.599967] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7770.606658] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7770.630273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7770.638214] Lustre: Skipped 1 previous similar message [ 7782.368669] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7782.664884] Lustre: server umount lustre-MDT0000 complete [ 7786.035749] LustreError: 254599:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789050297 with bad export cookie 4316994926336069571 [ 7786.041931] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7786.048344] LustreError: 254599:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7788.010723] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7788.022332] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7788.033192] Lustre: Skipped 4 previous similar messages [ 7788.057926] Lustre: Skipped 9 previous similar messages [ 7792.588663] Lustre: server umount lustre-MDT0001 complete [ 7803.108557] Lustre: server umount lustre-OST0000 complete [ 7822.304132] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7822.646095] Lustre: server umount lustre-OST0001 complete [ 7829.705419] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 7839.736110] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7875.088909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7875.451564] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7875.528891] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7875.667699] Lustre: lustre-MDT0000: new disk, initializing [ 7875.793405] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7875.801951] Lustre: Skipped 3 previous similar messages [ 7875.820262] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7880.439504] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7890.691628] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7890.829700] Lustre: 257784:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7890.918262] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7890.931116] Lustre: Skipped 1 previous similar message [ 7891.108917] Lustre: lustre-MDT0001: new disk, initializing [ 7891.245049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7891.253701] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7895.552279] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7905.286579] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7905.544852] Lustre: lustre-OST0000: new disk, initializing [ 7905.547579] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7905.552590] Lustre: 259365:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7906.937885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7906.957740] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7907.091919] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7910.648815] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7921.332464] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7921.558678] Lustre: lustre-OST0001: new disk, initializing [ 7921.562673] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7921.571174] Lustre: 260234:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7922.843480] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7922.860952] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7923.020869] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7927.956845] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7939.573180] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:27:29 (1789050449) [ 7954.790168] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:27:44 (1789050464) [ 7969.264156] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7969.272397] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7969.279871] Lustre: Skipped 1 previous similar message [ 7969.290117] Lustre: Skipped 4 previous similar messages [ 7975.172965] Lustre: server umount lustre-OST0000 complete [ 7979.499946] LustreError: 259347:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7979.524820] LustreError: 259347:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 17 previous similar messages [ 7985.031529] Lustre: server umount lustre-MDT0000 complete [ 7988.325724] LustreError: 257777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789050500 with bad export cookie 4316994926336071349 [ 7988.327666] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7988.335204] LustreError: 257777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7994.758846] Lustre: server umount lustre-MDT0001 complete [ 8004.372882] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8079.094618] Lustre: server umount lustre-OST0001 complete [ 8085.254360] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 8099.993814] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8134.783282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8135.020160] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8135.046336] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8135.105886] Lustre: lustre-MDT0000: new disk, initializing [ 8135.157542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8135.161239] Lustre: Skipped 3 previous similar messages [ 8135.170441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8139.760689] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8149.140147] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8149.210786] Lustre: 265091:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8149.589633] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8154.417735] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8163.962717] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8169.279911] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8176.769411] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8177.098404] Lustre: lustre-OST0000: new disk, initializing [ 8177.101468] Lustre: Skipped 1 previous similar message [ 8177.107804] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8177.120908] Lustre: Skipped 2 previous similar messages [ 8177.138178] Lustre: 267060:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8178.434529] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8178.443403] Lustre: Skipped 1 previous similar message [ 8178.448402] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8178.568891] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8184.116968] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8195.273285] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8204.261421] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8204.262338] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8204.275291] LustreError: Skipped 1 previous similar message [ 8204.276715] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8204.276722] Lustre: Skipped 11 previous similar messages [ 8204.313274] Lustre: Skipped 5 previous similar messages [ 8208.720314] Lustre: server umount lustre-OST0000 complete [ 8227.296331] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8227.462745] Lustre: server umount lustre-MDT0000 complete [ 8230.725651] LustreError: 265082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789050742 with bad export cookie 4316994926336072819 [ 8230.737901] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8231.128606] Lustre: server umount lustre-MDT0001 complete [ 8241.791651] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8244.590819] Key type lgssc unregistered [ 8244.919226] LNet: 268820:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8244.925421] LNetError: 268820:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8245.990672] LNet: Removed LNI 192.168.203.150@tcp [ 8247.041157] Key type .llcrypt unregistered [ 8247.045900] Key type ._llcrypt unregistered [ 8274.112759] Key type ._llcrypt registered [ 8274.114918] Key type .llcrypt registered [ 8274.232182] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8274.967056] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8275.040081] alg: No test for adler32 (adler32-zlib) [ 8276.098464] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 8276.391438] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 8278.040139] Key type lgssc registered [ 8279.178515] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8286.176313] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8286.210540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8287.628391] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8291.513750] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8297.953962] LustreError: 270280:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8297.968881] LustreError: 270280:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8299.013092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8299.359074] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8303.484167] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8310.654857] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8317.120665] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8324.406526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8324.958654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8332.161177] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8332.186648] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8340.279223] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8341.451908] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8346.593374] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8346.599938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8346.611323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8347.621339] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8347.627208] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8347.642114] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8352.746557] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8352.758939] Lustre: Skipped 1 previous similar message [ 8357.864440] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8357.889899] Lustre: Skipped 1 previous similar message [ 8358.368152] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8358.545796] Lustre: server umount lustre-OST0000 complete [ 8362.993886] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8363.000838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8363.016022] Lustre: Skipped 1 previous similar message [ 8363.022743] Lustre: Skipped 1 previous similar message [ 8368.931398] Lustre: server umount lustre-MDT0000 complete [ 8372.633324] LustreError: 270261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789050884 with bad export cookie 2203533007891802484 [ 8372.644831] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8373.072795] Lustre: server umount lustre-MDT0001 complete [ 8383.455295] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8386.115184] Key type lgssc unregistered [ 8386.502443] LNet: 273863:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8386.512075] LNetError: 273863:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8386.532272] LNet: Removed LNI 192.168.203.150@tcp [ 8387.398138] Key type .llcrypt unregistered [ 8387.399966] Key type ._llcrypt unregistered [ 8403.133197] Key type ._llcrypt registered [ 8403.135048] Key type .llcrypt registered [ 8403.236985] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8404.259656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8404.393599] alg: No test for adler32 (adler32-zlib) [ 8405.553648] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 8405.897173] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 8407.648255] Key type lgssc registered [ 8408.630574] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8416.894403] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8416.948538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8432.609955] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8437.729483] LustreError: 275056:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.150@tcp: failed processing log, type 4: rc = -110 [ 8464.480453] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8470.678131] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8479.423588] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8481.521899] Lustre: server umount lustre-OST0000 complete [ 8499.003347] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8501.978728] Key type lgssc unregistered [ 8502.237329] LNet: 276685:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8502.244077] LNetError: 276685:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8502.259812] LNet: Removed LNI 192.168.203.150@tcp [ 8503.291631] Key type .llcrypt unregistered [ 8503.299043] Key type ._llcrypt unregistered [ 8542.957299] Key type ._llcrypt registered [ 8542.961436] Key type .llcrypt registered [ 8543.046440] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8544.149287] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8544.175881] alg: No test for adler32 (adler32-zlib) [ 8545.211511] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 8545.437799] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 8547.112507] Key type lgssc registered [ 8548.112739] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8559.673468] loop0: detected capacity change from 0 to 204800000 [ 8559.859301] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8559.868039] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8560.317266] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8561.359790] loop0: detected capacity change from 0 to 204800000 [ 8561.554890] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8561.572812] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8561.922949] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8562.885185] loop0: detected capacity change from 0 to 204800000 [ 8563.095578] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8563.116344] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8563.298575] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8564.483842] loop0: detected capacity change from 0 to 204800000 [ 8564.661026] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8565.802579] loop0: detected capacity change from 0 to 204800000 [ 8565.839392] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8569.595428] loop0: detected capacity change from 0 to 204800000 [ 8569.636315] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8573.228511] loop0: detected capacity change from 0 to 204800000 [ 8573.245928] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8577.116687] loop0: detected capacity change from 0 to 204800000 [ 8577.162579] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8581.301975] loop0: detected capacity change from 0 to 204800000 [ 8581.433326] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8581.492013] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8581.559868] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8581.567282] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8581.576799] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8581.585469] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8581.593435] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8581.601219] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8581.612898] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8581.623370] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8581.631323] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8581.639656] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8581.649785] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8581.658547] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8581.667670] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8581.677212] Lustre: 280094:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8587.215368] Lustre: server umount MGS complete [ 8587.881290] loop0: detected capacity change from 0 to 204800000 [ 8587.937955] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8588.435618] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8588.467542] Lustre: lustre-MDT0000: Aborting client recovery [ 8589.317296] loop1: detected capacity change from 0 to 204800000 [ 8589.428436] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8589.522220] Lustre: 280526:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8589.528943] Lustre: 280526:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8589.546349] Lustre: 280526:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8589.564461] Lustre: 280526:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8589.587641] Lustre: 280526:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8589.598269] Lustre: 280526:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8589.613837] Lustre: 280526:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8589.660364] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8589.847028] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8589.897897] Lustre: lustre-MDT0001: Aborting client recovery [ 8590.729262] loop2: detected capacity change from 0 to 204800000 [ 8590.841873] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8590.916587] Lustre: 280605:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8590.934272] Lustre: 280605:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8590.946783] Lustre: 280605:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8590.967505] Lustre: 280605:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8590.977329] Lustre: 280605:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8590.989438] Lustre: 280605:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8590.999148] Lustre: 280605:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8591.098241] Lustre: Found index 0 for lustre-OST0000, updating log [ 8591.138546] Lustre: 280623:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8591.186835] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8591.210142] Lustre: lustre-OST0000: Aborting recovery [ 8592.009096] loop3: detected capacity change from 0 to 204800000 [ 8592.115938] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8592.241212] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8592.254196] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8592.265543] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8592.273914] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8592.282681] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8592.295948] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8592.307595] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8592.333482] Lustre: Found index 1 for lustre-OST0001, updating log [ 8592.372687] Lustre: 280694:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8592.467357] Lustre: lustre-OST0001: Aborting recovery [ 8597.496829] LustreError: 280470:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8598.509919] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8598.514398] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8598.550484] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8598.560301] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8599.539399] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8599.543093] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8600.037874] LustreError: 280436:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 535us: rc = -2 [ 8600.539318] LustreError: 280433:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 644us: rc = -2 [ 8600.557517] LustreError: 280433:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 30 previous similar messages [ 8614.881371] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8614.884175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8614.920343] Lustre: Skipped 3 previous similar messages [ 8614.936908] Lustre: Skipped 3 previous similar messages [ 8619.004863] Lustre: server umount lustre-MDT0000 complete [ 8619.815530] LustreError: 280420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789051131 with bad export cookie 11069495049435325272 [ 8619.825584] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8620.000684] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8620.002302] LustreError: 280434:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8620.002470] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8620.022586] Lustre: Skipped 1 previous similar message [ 8620.047646] LustreError: 280434:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8625.123638] LustreError: 280435:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8625.127278] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8625.144626] Lustre: Skipped 2 previous similar messages [ 8629.221544] LustreError: 280433:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8629.221959] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8629.261468] LustreError: 280433:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8629.307551] Lustre: Skipped 1 previous similar message [ 8634.336383] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8634.336553] LustreError: 280435:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8634.336681] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8634.401475] LustreError: 280435:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8634.757588] Lustre: server umount lustre-MDT0001 complete [ 8650.208169] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8650.375083] Lustre: server umount lustre-OST0000 complete [ 8657.383860] Lustre: server umount lustre-OST0001 complete [ 8664.389828] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 10:39:34 (1789051174) [ 8690.722311] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 8699.889382] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8736.328860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8736.534481] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8736.564322] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8736.647363] Lustre: lustre-MDT0000: new disk, initializing [ 8736.723723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8736.733649] Lustre: Skipped 1 previous similar message [ 8736.750939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8741.105680] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8751.590980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8751.815274] Lustre: 284766:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8751.872941] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8751.876943] Lustre: Skipped 1 previous similar message [ 8751.965490] Lustre: lustre-MDT0001: new disk, initializing [ 8752.098519] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8752.162348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8752.185611] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8756.944525] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8767.038697] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8773.923978] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8781.128331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8781.510479] Lustre: lustre-OST0000: new disk, initializing [ 8781.513164] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8781.517745] Lustre: 286733:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8781.602946] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8787.767912] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8791.189060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8791.232190] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8791.307486] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8796.813796] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8802.426762] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8812.047774] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8818.109772] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8824.983853] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8868.335293] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8868.337503] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8868.341444] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8868.341451] Lustre: Skipped 1 previous similar message [ 8868.341947] LustreError: Skipped 1 previous similar message [ 8868.379351] Lustre: Skipped 1 previous similar message [ 8882.145049] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8882.311218] Lustre: server umount lustre-OST0000 complete [ 8888.801521] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8888.833156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8888.840615] Lustre: Skipped 5 previous similar messages [ 8892.830527] Lustre: server umount lustre-MDT0000 complete [ 8893.923417] LustreError: 284777:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8893.950963] LustreError: 284777:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8896.997956] LustreError: 284756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789051408 with bad export cookie 11069495049435327974 [ 8897.003576] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8897.025313] LustreError: 284756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8897.528575] Lustre: server umount lustre-MDT0001 complete [ 8908.563692] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8913.477095] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8918.119978] Lustre: server umount MGS complete [ 8929.379313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8929.896707] LustreError: 290557:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8929.946334] LustreError: 290570:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8930.049034] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8934.663427] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8943.691243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8950.193432] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8958.490920] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8963.946993] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8970.805513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8971.300736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8971.316967] Lustre: Skipped 1 previous similar message [ 8978.379802] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8986.191216] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9015.266235] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9015.271964] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9015.290502] LustreError: Skipped 1 previous similar message [ 9015.292856] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9015.321638] Lustre: Skipped 2 previous similar messages [ 9020.907567] Lustre: server umount lustre-OST0000 complete [ 9025.507687] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9025.521081] Lustre: Skipped 1 previous similar message [ 9039.328403] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9039.525118] Lustre: server umount lustre-MDT0000 complete [ 9040.865774] LustreError: 290552:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9040.887299] LustreError: 290552:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 9043.045329] LustreError: 290538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789051554 with bad export cookie 11069495049435329927 [ 9043.049239] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9043.059469] LustreError: 290538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9043.393311] Lustre: server umount lustre-MDT0001 complete [ 9053.905806] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 9056.705406] Key type lgssc unregistered [ 9057.032491] LNet: 294499:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9057.043597] LNetError: 294499:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9057.060141] LNet: Removed LNI 192.168.203.150@tcp [ 9057.926325] Key type .llcrypt unregistered [ 9057.928453] Key type ._llcrypt unregistered [ 9092.411262] Key type ._llcrypt registered [ 9092.413864] Key type .llcrypt registered [ 9092.522193] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 9107.986168] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 9109.414489] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9109.601829] alg: No test for adler32 (adler32-zlib) [ 9110.883263] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 9111.217280] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 9113.010888] Key type lgssc registered [ 9114.687595] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9150.834323] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9150.868932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9152.078561] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9152.100404] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9152.219135] Lustre: lustre-MDT0000: new disk, initializing [ 9152.310700] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9152.322847] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9156.796625] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9167.452639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9167.600616] Lustre: 298451:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9167.659283] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9167.664881] Lustre: Skipped 1 previous similar message [ 9167.791456] Lustre: lustre-MDT0001: new disk, initializing [ 9167.903176] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9167.957076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9167.970652] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9172.793979] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9183.902730] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9189.764619] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9197.113136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9197.494963] Lustre: lustre-OST0000: new disk, initializing [ 9197.500047] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9197.511388] Lustre: 300417:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9197.660258] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9204.645424] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9205.835942] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9205.850431] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9205.968820] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9215.084633] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9223.780429] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9233.971664] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9243.527555] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9255.234762] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9298.403745] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9298.407234] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9298.410382] LustreError: Skipped 1 previous similar message [ 9298.411612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9298.448450] Lustre: Skipped 1 previous similar message [ 9300.827243] Lustre: server umount lustre-OST0000 complete [ 9306.085833] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9306.109884] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9306.123396] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9306.127654] Lustre: Skipped 1 previous similar message [ 9308.645805] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9308.648196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9308.682207] Lustre: Skipped 1 previous similar message [ 9313.763916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9313.777425] Lustre: Skipped 1 previous similar message [ 9318.882264] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9318.890326] Lustre: Skipped 1 previous similar message [ 9319.392792] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9319.714197] Lustre: server umount lustre-MDT0000 complete [ 9323.496949] LustreError: 301273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789051835 with bad export cookie 18134390983908914840 [ 9323.506139] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9323.511334] LustreError: 301273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9323.827604] Lustre: server umount lustre-MDT0001 complete [ 9331.595157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9336.019226] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9340.941978] Lustre: server umount MGS complete [ 9353.386243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9354.022844] LustreError: 304376:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9354.120825] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9359.077726] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9359.331348] LustreError: 304364:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9359.377199] LustreError: 304364:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9363.427797] LustreError: 304363:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9368.290098] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9368.546311] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 9368.837251] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9373.381401] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9379.975943] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9384.979618] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9392.788607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9393.326708] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9401.199707] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9409.343953] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9439.724506] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9439.738050] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9439.748351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9439.762731] Lustre: Skipped 1 previous similar message [ 9444.048275] Lustre: server umount lustre-OST0000 complete [ 9449.966697] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9449.985283] Lustre: Skipped 2 previous similar messages [ 9453.889679] Lustre: server umount lustre-MDT0000 complete [ 9455.086847] LustreError: 305408:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9458.004618] LustreError: 304345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789051969 with bad export cookie 18134390983908916779 [ 9458.013562] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9458.025289] LustreError: 304345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9458.592586] Lustre: server umount lustre-MDT0001 complete [ 9470.664801] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 9474.096680] Key type lgssc unregistered [ 9474.567152] LNet: 308308:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9474.575202] LNetError: 308308:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9475.624547] LNet: Removed LNI 192.168.203.150@tcp [ 9476.969813] Key type .llcrypt unregistered [ 9476.971753] Key type ._llcrypt unregistered [ 9500.512418] Key type ._llcrypt registered [ 9500.514683] Key type .llcrypt registered [ 9500.620146] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 9515.399106] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 9516.409821] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9516.447575] alg: No test for adler32 (adler32-zlib) [ 9517.771586] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [ 9518.032117] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 9519.744305] Key type lgssc registered [ 9520.963069] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9549.631618] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 9564.972121] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9564.996986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9566.167424] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9566.196707] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9566.297485] Lustre: lustre-MDT0000: new disk, initializing [ 9566.391918] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9566.408136] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9571.130765] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9581.235531] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9581.473954] Lustre: lustre-OST0000: new disk, initializing [ 9581.478479] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9581.484538] Lustre: Skipped 1 previous similar message [ 9581.489919] Lustre: 312357:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9581.564284] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9583.063352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9583.071432] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9583.098397] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9587.555532] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10451.426672] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10451.453694] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10456.547439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10458.839798] Lustre: server umount lustre-MDT0000 complete [10462.688776] LustreError: 311545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789052974 with bad export cookie 16930588149709166104 [10462.692422] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10462.701103] LustreError: 311545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10462.926110] Lustre: server umount lustre-OST0000 complete [10490.991234] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [10504.207377] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [10553.607726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10554.172883] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10554.290044] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10554.552330] Lustre: lustre-MDT0000: new disk, initializing [10554.768507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10554.822924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10562.492706] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10575.675513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10575.794215] Lustre: 316708:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10575.849221] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10575.855847] Lustre: Skipped 1 previous similar message [10575.909268] Lustre: lustre-MDT0001: new disk, initializing [10576.025886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10576.069378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10576.093107] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10581.350811] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10590.261546] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10596.372615] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10602.751844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10602.922721] Lustre: lustre-OST0000: new disk, initializing [10602.926170] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10602.931225] Lustre: 318677:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10603.011624] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10608.401531] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10611.831204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10611.839440] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10611.949778] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10619.370973] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10624.315163] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10624.570812] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10629.306102] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10629.505154] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10632.673425] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10632.678506] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10632.685232] LustreError: Skipped 1 previous similar message [10632.686371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10632.726681] Lustre: Skipped 1 previous similar message [10637.650816] Lustre: server umount lustre-OST0000 complete [10642.914976] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10642.918760] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10642.932100] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10642.932110] Lustre: Skipped 1 previous similar message [10642.963050] Lustre: Skipped 1 previous similar message [10648.150863] Lustre: server umount lustre-MDT0000 complete [10652.701733] LustreError: 316701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789053164 with bad export cookie 16930588149712106923 [10652.705068] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10652.729389] LustreError: 316701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10653.239265] Lustre: server umount lustre-MDT0001 complete [10657.531352] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10660.036276] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:12:49 (1789053169) [10669.419270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10670.245183] LustreError: 320992:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10670.282511] LustreError: 320992:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [10670.395738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10675.213742] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10675.683339] LustreError: 320993:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10680.806941] LustreError: 320992:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10685.457761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10686.063793] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10691.802701] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10700.682387] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10706.189728] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10713.859533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10714.333337] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10720.908868] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10728.717105] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10735.847306] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10736.133384] Lustre: lustre-OST0001: new disk, initializing [10736.137970] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10736.144166] Lustre: 323828:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10736.286611] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10741.892705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10741.906575] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10741.967891] LustreError: 324125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10742.001327] LustreError: 324125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10743.018314] LustreError: 324125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10743.044539] LustreError: 324125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10743.387828] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10744.039428] LustreError: 324129:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10746.084634] LustreError: 324129:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10746.102110] LustreError: 324129:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10750.246560] LustreError: 324129:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10750.257046] LustreError: 324129:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10754.535708] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10754.731922] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10758.565212] LustreError: 324129:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10758.575787] LustreError: 324129:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10761.984783] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0723b53c800.ost_server_uuid 50 [10763.275837] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0723b53c800.ost_server_uuid in FULL state after 0 sec [10769.158738] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0723b53c800.ost_server_uuid 50 [10770.683301] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0723b53c800.ost_server_uuid in FULL state after 0 sec [10778.113142] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10788.333604] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10788.348276] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10788.350657] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10788.372262] LustreError: Skipped 1 previous similar message [10788.385618] Lustre: Skipped 2 previous similar messages [10792.214575] Lustre: server umount lustre-OST0001 complete [10793.451673] LustreError: 325364:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10793.474953] LustreError: 325364:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10798.585487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10798.606720] LustreError: 322816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10798.606731] LustreError: 322816:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10798.608340] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10798.608347] Lustre: Skipped 1 previous similar message [10798.652902] Lustre: Skipped 2 previous similar messages [10804.828955] Lustre: server umount lustre-OST0000 complete [10813.921475] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10813.929721] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10813.954048] Lustre: Skipped 1 previous similar message [10819.041399] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10819.052958] Lustre: Skipped 6 previous similar messages [10823.138433] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10823.544920] Lustre: server umount lustre-MDT0000 complete [10824.164352] LustreError: 320993:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10824.185116] LustreError: 320993:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [10827.216024] LustreError: 320973:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789053339 with bad export cookie 16930588149712107805 [10827.225116] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10827.247199] LustreError: 320973:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10827.614941] Lustre: server umount lustre-MDT0001 complete [10838.140682] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [10841.104539] Key type lgssc unregistered [10841.440133] LNet: 326521:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10841.448314] LNetError: 326521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10842.470523] LNet: Removed LNI 192.168.203.150@tcp [10843.367841] Key type .llcrypt unregistered [10843.371589] Key type ._llcrypt unregistered [10868.560362] Key type ._llcrypt registered [10868.562815] Key type .llcrypt registered [10868.642545] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [10869.574795] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10869.722795] alg: No test for adler32 (adler32-zlib) [10871.026901] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [10871.275523] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [10872.993130] Key type lgssc registered [10874.285880] Lustre: Echo OBD driver; http://www.lustre.org/ [10881.848664] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10881.873287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10883.335630] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10887.552338] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10893.794195] LustreError: 327940:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10893.819603] LustreError: 327940:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10894.260409] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10898.915458] LustreError: 327939:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10901.313835] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10901.660133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10906.470835] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10916.430506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10917.084637] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10923.871126] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10926.309034] LustreError: 329620:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10928.356446] LustreError: 329620:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10928.388489] LustreError: 329620:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10928.402250] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10931.850660] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10933.734535] LustreError: 329620:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10933.757065] LustreError: 329620:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10938.754786] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10938.974491] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10943.994738] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10945.315730] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10953.233885] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10959.565384] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10959.801057] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11071.978449] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11071.981140] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11071.988660] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11072.017605] Lustre: Skipped 1 previous similar message [11077.090591] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11077.105520] Lustre: Skipped 2 previous similar messages [11077.739381] Lustre: server umount lustre-OST0001 complete [11082.221225] LustreError: 329621:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11082.242186] LustreError: 329621:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11085.290363] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11085.301301] LustreError: Skipped 1 previous similar message [11085.315193] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11085.341346] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11087.329647] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11087.336983] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11087.350096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11087.353621] Lustre: Skipped 1 previous similar message [11091.511547] Lustre: server umount lustre-OST0000 complete [11096.036211] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11096.055500] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11096.070464] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11102.239398] Lustre: server umount lustre-MDT0000 complete [11102.689411] LustreError: 327939:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11102.711525] LustreError: 327939:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11106.232703] LustreError: 327918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789053618 with bad export cookie 4986237888249968034 [11106.239488] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11106.245186] LustreError: 327918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11106.781755] Lustre: server umount lustre-MDT0001 complete [11118.709401] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [11121.910273] Key type lgssc unregistered [11122.372395] LNet: 332887:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11122.392507] LNetError: 332887:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11122.429637] LNet: Removed LNI 192.168.203.150@tcp [11123.553333] Key type .llcrypt unregistered [11123.557503] Key type ._llcrypt unregistered [11169.061811] Key type ._llcrypt registered [11169.064691] Key type .llcrypt registered [11169.166597] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11170.222641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11170.300792] alg: No test for adler32 (adler32-zlib) [11171.368609] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [11171.548865] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [11173.241701] Key type lgssc registered [11174.436715] Lustre: Echo OBD driver; http://www.lustre.org/ [11183.337680] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11183.355208] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11184.932287] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11189.577252] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11194.347379] LustreError: 335175:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11194.374483] LustreError: 335175:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11197.423412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11197.789213] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11201.546934] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11204.621155] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11210.325953] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11210.653798] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11216.660926] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11222.898205] LustreError: 336665:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11226.737387] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11227.187657] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11229.266484] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11229.273612] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11233.080427] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11244.743861] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11251.168108] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11269.601597] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11269.631573] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11269.689952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11274.481716] Lustre: server umount lustre-MDT0000 complete [11275.238391] LustreError: 337880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11275.261317] LustreError: 337880:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11280.363224] LustreError: 335175:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11280.392855] LustreError: 335175:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11280.399831] LustreError: 335154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789053792 with bad export cookie 1531645926706840998 [11280.403782] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11280.438546] LustreError: 335154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11280.911317] Lustre: server umount lustre-MDT0001 complete [11296.834143] Lustre: server umount lustre-OST0000 complete [11311.812275] Lustre: server umount lustre-OST0001 complete [11333.661126] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11346.263420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11346.677129] LustreError: 340443:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11346.767091] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11352.108520] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11356.138085] LustreError: 340444:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11356.162928] LustreError: 340444:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11362.631709] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11363.255265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11368.808761] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11372.067273] Lustre: 341586:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11380.340340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11380.842274] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11389.063022] LustreError: 341940:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11389.085519] LustreError: 341940:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11389.989650] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11391.152996] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11401.802393] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11407.370779] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11410.834303] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11419.006757] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11422.912953] Lustre: 343458:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11427.810406] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11427.819735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11427.827426] Lustre: Skipped 6 previous similar messages [11427.838669] Lustre: Skipped 7 previous similar messages [11432.933482] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11432.943028] Lustre: Skipped 1 previous similar message [11433.456454] Lustre: server umount lustre-MDT0000 complete [11437.769463] LustreError: 340425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789053949 with bad export cookie 1531645926706842699 [11437.787151] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11437.793223] LustreError: 340425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11438.056795] LustreError: 342948:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11438.059361] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11438.080531] LustreError: 342948:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [11438.085383] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11438.095715] Lustre: Skipped 1 previous similar message [11438.103584] Lustre: Skipped 1 previous similar message [11438.337825] Lustre: server umount lustre-MDT0001 complete [11444.924608] Lustre: server umount lustre-OST0000 complete [11448.775961] Lustre: server umount lustre-OST0001 complete [11453.908932] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11455.552259] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11457.828354] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:26:07 (1789053967) [11486.033775] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11497.171042] loop0: detected capacity change from 0 to 1099511627776 [11497.884969] SGI XFS with ACLs, security attributes, quota, no debug enabled [11497.899039] *** VALIDATE xfs *** [11497.910965] XFS (loop0): Mounting V5 Filesystem [11497.940806] XFS (loop0): Ending clean mount [11499.412303] loop1: detected capacity change from 0 to 18691697672192 [11609.528836] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [11620.380985] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11655.448879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11655.815655] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11655.859629] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11655.968570] Lustre: lustre-MDT0000: new disk, initializing [11656.101515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11656.113517] Lustre: Skipped 1 previous similar message [11656.139462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11662.239094] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11673.988315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11674.143071] Lustre: 348590:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11674.203757] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11674.207873] Lustre: Skipped 1 previous similar message [11674.297867] Lustre: lustre-MDT0001: new disk, initializing [11674.475299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11674.500403] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11679.048574] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11688.338132] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11693.720773] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11699.378822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11699.628811] Lustre: lustre-OST0000: new disk, initializing [11699.633498] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11699.641693] Lustre: 350558:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11700.940044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11700.954930] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11701.054543] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11705.034255] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11713.631092] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11718.137233] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11718.327787] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11722.155678] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11722.331600] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11726.818266] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11726.825254] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11726.832672] LustreError: Skipped 1 previous similar message [11726.833329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11726.833333] Lustre: Skipped 1 previous similar message [11726.877259] Lustre: Skipped 1 previous similar message [11730.395941] Lustre: server umount lustre-OST0000 complete [11736.035975] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11736.044098] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11736.060534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11736.063318] Lustre: Skipped 1 previous similar message [11740.592444] Lustre: server umount lustre-MDT0000 complete [11742.180822] LustreError: 348603:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11742.192893] LustreError: 348603:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11744.362529] LustreError: 348584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789054256 with bad export cookie 1531645926706844246 [11744.371159] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11744.383744] LustreError: 348584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11744.783303] Lustre: server umount lustre-MDT0001 complete [11754.000930] XFS (loop0): Unmounting Filesystem [11759.425406] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:31:08 (1789054268) [11765.939354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11766.536464] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11766.547090] Lustre: Skipped 2 previous similar messages [11770.935825] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11780.524745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11786.179464] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11793.951396] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11799.547640] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11806.409785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11813.283123] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11820.746674] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11836.392682] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11836.399643] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11836.441232] Lustre: Skipped 1 previous similar message [11836.453274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11836.466699] Lustre: Skipped 2 previous similar messages [11841.914420] Lustre: server umount lustre-OST0000 complete [11847.649272] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11853.873793] Lustre: server umount lustre-MDT0000 complete [11857.244653] LustreError: 353101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789054369 with bad export cookie 1531645926706845128 [11857.269776] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11857.651634] Lustre: server umount lustre-MDT0001 complete [11869.553454] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [11873.101141] Key type lgssc unregistered [11873.473160] LNet: 356748:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11873.495454] LNetError: 356748:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11873.532232] LNet: Removed LNI 192.168.203.150@tcp [11874.764379] Key type .llcrypt unregistered [11874.772568] Key type ._llcrypt unregistered [11901.142593] Key type ._llcrypt registered [11901.144938] Key type .llcrypt registered [11901.261281] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11902.168335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11902.337033] alg: No test for adler32 (adler32-zlib) [11903.477365] Lustre: Lustre: Build Version: 2.17.58_2_g725efe4 [11903.828027] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [11905.520432] Key type lgssc registered [11906.935778] Lustre: Echo OBD driver; http://www.lustre.org/ [11913.873482] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11913.894886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11915.209454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11919.822313] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11925.488143] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11925.532314] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11928.812641] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11929.498497] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11935.536911] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11945.151111] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11950.893390] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11957.715708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11957.994497] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11965.629041] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11974.556396] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11984.876975] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11984.887091] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11984.889699] Lustre: Skipped 1 previous similar message [11987.804128] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.50@tcp (stopping) [11989.329487] Lustre: server umount lustre-MDT0001 complete [11989.992996] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11990.007447] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11992.891783] LustreError: 360521:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11995.106795] LustreError: 358310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11995.145234] LustreError: 358310:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12000.227721] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12000.252770] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12004.296242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12004.482477] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12004.498957] Lustre: Found index 1 for lustre-MDT0001, updating log [12004.563860] Lustre: 358303:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12004.887295] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12010.000459] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12012.016527] LustreError: 361735:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12012.032670] LustreError: 361735:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12012.045715] LustreError: 358305:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12012.152796] LustreError: 361746:0:(obd_config.c:2200:class_config_llog_handler()) MGC192.168.203.150@tcp: cfg command failed: rc = -17 [12012.174228] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12012.174228] [12012.361970] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12025.316099] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12025.320754] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12025.341351] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12025.369553] Lustre: Skipped 2 previous similar messages [12025.392903] Lustre: Skipped 1 previous similar message [12028.772138] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.50@tcp (stopping) [12031.055492] Lustre: server umount lustre-MDT0001 complete [12033.845883] LustreError: 360521:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12033.887617] LustreError: 360521:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12043.221427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12043.298698] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12043.310683] Lustre: Found index 1 for lustre-MDT0001, updating log [12043.367611] Lustre: 358301:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12043.759018] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12048.537716] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12048.896226] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12050.965389] LustreError: 362958:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12050.971490] LustreError: 362958:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12050.987675] LustreError: 358305:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12050.995401] LustreError: 358305:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12051.102569] LustreError: 362965:0:(obd_config.c:827:class_setup()) osp: device 8 already setup: rc = -17 [12051.109342] LustreError: 362965:0:(obd_config.c:941:class_detach()) OBD device 8 still set up [12051.124426] LustreError: 362965:0:(obd_config.c:2200:class_config_llog_handler()) MGC192.168.203.150@tcp: cfg command failed: rc = -17 [12051.142077] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.150@tcp [12051.142077] [12356.065221] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12356.073104] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12356.087045] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12356.096122] Lustre: Skipped 2 previous similar messages [12359.149438] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12364.265678] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12364.275658] Lustre: Skipped 5 previous similar messages [12369.381558] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12369.589196] Lustre: server umount lustre-MDT0001 complete [12371.771773] LustreError: 358309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12371.797184] LustreError: 358309:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12379.475243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12379.635398] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12379.647997] Lustre: Found index 1 for lustre-MDT0001, updating log [12379.692503] Lustre: 358301:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12379.996231] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12384.228652] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12385.263903] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12385.309943] LustreError: 371613:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12385.317483] LustreError: 371613:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12385.324081] LustreError: 358305:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12385.333904] LustreError: 358305:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12385.433472] LustreError: 371622:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12385.458101] LustreError: 371622:0:(obd_config.c:2200:class_config_llog_handler()) MGC192.168.203.150@tcp: cfg command failed: rc = -114 [12385.473379] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12385.473379] [12707.812994] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12707.822252] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12707.855238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12707.864452] Lustre: Skipped 3 previous similar messages [12712.640131] Lustre: server umount lustre-MDT0000 complete [12716.017140] LustreError: 358309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12716.037705] LustreError: 358309:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [12716.380806] LustreError: 361499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789055228 with bad export cookie 11166774753156743559 [12716.385589] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12716.393446] LustreError: 361499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12721.130693] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12721.149316] Lustre: Skipped 2 previous similar messages [12722.829927] Lustre: server umount lustre-MDT0001 complete [12726.741772] Lustre: server umount lustre-OST0000 complete [12735.339041] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [12744.956064] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [12781.751296] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 11:48:11 (1789055291) [12789.002841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12789.258646] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12789.283253] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12789.400472] Lustre: lustre-MDT0000: new disk, initializing [12789.513443] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12789.529964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12793.890404] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12804.344139] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12804.470825] Lustre: 382251:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12804.512584] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12804.515409] Lustre: Skipped 1 previous similar message [12804.571473] Lustre: lustre-MDT0001: new disk, initializing [12804.663795] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12804.719058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12804.736374] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12809.143115] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12817.935898] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12823.377871] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12830.492426] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12830.774379] Lustre: lustre-OST0000: new disk, initializing [12830.777983] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12830.785855] Lustre: 384217:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12830.874301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12836.429403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12836.451530] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12836.569373] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12836.919446] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12845.475853] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12853.754273] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12853.761713] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12853.778523] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12853.781367] Lustre: Skipped 4 previous similar messages [12855.358246] Lustre: server umount lustre-MDT0000 complete [12857.318473] LustreError: 383467:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12857.338500] LustreError: 383467:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12858.850968] LustreError: 382241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789055370 with bad export cookie 11166774753156745512 [12858.852798] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12858.866775] LustreError: 382241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12858.880900] LustreError: 383057:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12859.191682] Lustre: server umount lustre-MDT0001 complete [12873.209614] Lustre: server umount lustre-OST0000 complete [12904.895719] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [12916.844408] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [12952.076813] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12591 sec ========= 11:51:02 (1789055462) [12954.555771] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 11:51:03 (1789055463) === [12957.997302] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 11:51:07 (1789055467) === [13001.435341] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [13004.970460] Key type lgssc unregistered [13005.245676] LNet: 390582:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13005.253680] LNetError: 390582:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13005.271256] LNet: Removed LNI 192.168.203.150@tcp [13006.134183] Key type .llcrypt unregistered [13006.138193] Key type ._llcrypt unregistered