[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 529286510 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.996 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003306] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229833f6470, max_idle_ns: 440795327230 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399996) [ 0.009016] pid_max: default: 32768 minimum: 301 [ 0.010133] LSM: Security Framework initializing [ 0.011049] Yama: becoming mindful. [ 0.012032] SELinux: Initializing. [ 0.013062] *** VALIDATE selinux *** [ 0.021125] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025523] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026150] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027109] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029042] *** VALIDATE tmpfs *** [ 0.030406] *** VALIDATE proc *** [ 0.031229] *** VALIDATE cgroup *** [ 0.032007] *** VALIDATE cgroup2 *** [ 0.033249] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034129] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036036] Spectre V2 : User space: Vulnerable [ 0.037006] Speculative Store Bypass: Vulnerable [ 0.040234] debug: unmapping init [mem 0xffffffffa2659000-0xffffffffa2660fff] [ 0.043182] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044701] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045025] ... version: 2 [ 0.046008] ... bit width: 48 [ 0.047011] ... generic registers: 4 [ 0.048008] ... value mask: 0000ffffffffffff [ 0.049012] ... max period: 00007fffffffffff [ 0.050011] ... fixed-purpose events: 3 [ 0.050951] ... event mask: 000000070000000f [ 0.051315] rcu: Hierarchical SRCU implementation. [ 0.053442] smp: Bringing up secondary CPUs ... [ 0.054518] x86: Booting SMP configuration: [ 0.055023] .... node #0, CPUs: #1 #2 #3 [ 0.058015] smp: Brought up 1 node, 4 CPUs [ 0.060014] smpboot: Max logical packages: 1 [ 0.061020] smpboot: Total of 4 processors activated (19199.96 BogoMIPS) [ 0.100408] node 0 deferred pages initialised in 37ms [ 0.104097] devtmpfs: initialized [ 0.105285] x86/mm: Memory block size: 128MB [ 0.108309] gcov: version magic: 0x41383552 [ 0.111266] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.114074] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.116437] pinctrl core: initialized pinctrl subsystem [ 0.119203] [ 0.119860] ************************************************************* [ 0.122012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.124009] ** ** [ 0.127013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.129010] ** ** [ 0.131009] ** This means that this kernel is built to expose internal ** [ 0.132008] ** IOMMU data structures, which may compromise security on ** [ 0.134010] ** your system. ** [ 0.135009] ** ** [ 0.137011] ** If you see this message and you are not debugging the ** [ 0.139010] ** kernel, report this immediately to your vendor! ** [ 0.141012] ** ** [ 0.144012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146011] ************************************************************* [ 0.148672] NET: Registered protocol family 16 [ 0.150402] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.153067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.155066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.158036] cpuidle: using governor menu [ 0.159707] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.160626] PCI: Using configuration type 1 for base access [ 0.161000] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.170107] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.172041] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.176032] cryptd: max_cpu_qlen set to 1000 [ 0.180208] ACPI: Added _OSI(Module Device) [ 0.182014] ACPI: Added _OSI(Processor Device) [ 0.184012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.185014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.190558] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.198751] ACPI: Interpreter enabled [ 0.200060] ACPI: PM: (supports S0 S3 S4 S5) [ 0.201010] ACPI: Using IOAPIC for interrupt routing [ 0.202121] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.205551] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.216897] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.219040] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.221022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.224089] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.230110] acpiphp: Slot [2] registered [ 0.231200] acpiphp: Slot [5] registered [ 0.233131] acpiphp: Slot [6] registered [ 0.234098] acpiphp: Slot [7] registered [ 0.236138] acpiphp: Slot [8] registered [ 0.237095] acpiphp: Slot [9] registered [ 0.239098] acpiphp: Slot [10] registered [ 0.240166] acpiphp: Slot [3] registered [ 0.242106] acpiphp: Slot [4] registered [ 0.243110] acpiphp: Slot [11] registered [ 0.245106] acpiphp: Slot [12] registered [ 0.247105] acpiphp: Slot [13] registered [ 0.247904] acpiphp: Slot [14] registered [ 0.248074] acpiphp: Slot [15] registered [ 0.250080] acpiphp: Slot [16] registered [ 0.251117] acpiphp: Slot [17] registered [ 0.253176] acpiphp: Slot [18] registered [ 0.254104] acpiphp: Slot [19] registered [ 0.256097] acpiphp: Slot [20] registered [ 0.257186] acpiphp: Slot [21] registered [ 0.259076] acpiphp: Slot [22] registered [ 0.260100] acpiphp: Slot [23] registered [ 0.261065] acpiphp: Slot [24] registered [ 0.262082] acpiphp: Slot [25] registered [ 0.264081] acpiphp: Slot [26] registered [ 0.265086] acpiphp: Slot [27] registered [ 0.266091] acpiphp: Slot [28] registered [ 0.267094] acpiphp: Slot [29] registered [ 0.269097] acpiphp: Slot [30] registered [ 0.270065] acpiphp: Slot [31] registered [ 0.271000] PCI host bridge to bus 0000:00 [ 0.271000] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.274020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.276016] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.278030] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.280033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.283027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.285212] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.287848] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.291298] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.301014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.306937] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.309018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.311020] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.314017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.318536] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.320679] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.322032] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.324712] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.330016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.342017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.347013] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.352065] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.359019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.367016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.389021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.400355] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.420035] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.435028] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.535040] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.574087] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.590016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.596017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.621018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.638230] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.680041] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.701025] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.773027] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.786262] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.798016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.804015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.820028] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.833300] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.841018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.848019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.865020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.875129] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.878446] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.881409] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.883389] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.885271] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.894049] iommu: Default domain type: Passthrough [ 0.895537] SCSI subsystem initialized [ 0.897206] ACPI: bus type USB registered [ 0.899117] usbcore: registered new interface driver usbfs [ 0.900075] usbcore: registered new interface driver hub [ 0.902159] usbcore: registered new device driver usb [ 0.905245] pps_core: LinuxPPS API ver. 1 registered [ 0.907010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.911103] PTP clock support registered [ 0.913198] EDAC MC: Ver: 3.0.0 [ 0.915165] PCI: Using ACPI for IRQ routing [ 0.917043] NetLabel: Initializing [ 0.919014] NetLabel: domain hash size = 128 [ 0.921014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.923101] NetLabel: unlabeled traffic allowed by default [ 0.925375] vgaarb: loaded [ 0.926405] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.928014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.934315] clocksource: Switched to clocksource kvm-clock [ 1.052935] VFS: Disk quotas dquot_6.6.0 [ 1.054369] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.057639] *** VALIDATE ramfs *** [ 1.058689] *** VALIDATE hugetlbfs *** [ 1.060171] pnp: PnP ACPI init [ 1.062403] pnp: PnP ACPI: found 6 devices [ 1.081200] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.083815] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.085544] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.087444] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.089570] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.091621] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.093924] NET: Registered protocol family 2 [ 1.096071] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.100819] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.104795] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.110658] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.113993] TCP: Hash tables configured (established 65536 bind 65536) [ 1.116703] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.119662] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.122532] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.125682] NET: Registered protocol family 1 [ 1.129752] RPC: Registered named UNIX socket transport module. [ 1.131657] RPC: Registered udp transport module. [ 1.133327] RPC: Registered tcp transport module. [ 1.134444] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.136447] NET: Registered protocol family 44 [ 1.137911] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.139601] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.141753] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.143624] PCI: CLS 0 bytes, default 64 [ 1.145884] Unpacking initramfs... [ 2.636743] debug: unmapping init [mem 0xffff982efcc54000-0xffff982efffbffff] [ 2.641100] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.643493] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.646218] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229833f6470, max_idle_ns: 440795327230 ns [ 3.126374] Initialise system trusted keyrings [ 3.127784] Key type blacklist registered [ 3.129927] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.139881] zbud: loaded [ 3.143326] *** VALIDATE nfs *** [ 3.144608] *** VALIDATE nfs4 *** [ 3.146168] pstore: using deflate compression [ 3.149707] Platform Keyring initialized [ 3.271086] NET: Registered protocol family 38 [ 3.272704] Key type asymmetric registered [ 3.274236] Asymmetric key parser 'x509' registered [ 3.275984] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.278308] io scheduler mq-deadline registered [ 3.279754] io scheduler kyber registered [ 3.281439] io scheduler bfq registered [ 3.283310] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.286542] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.289677] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.292623] ACPI: Power Button [PWRF] [ 3.297828] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.304318] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.348747] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.356209] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.370162] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.396994] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.422537] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.426875] Non-volatile memory driver v1.3 [ 3.428702] Linux agpgart interface v0.103 [ 3.459528] virtio_blk virtio1: [vda] 145168 512-byte logical blocks (74.3 MB/70.9 MiB) [ 3.462749] vda: detected capacity change from 0 to 74326016 [ 3.510688] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.513652] vdb: detected capacity change from 0 to 1073741824 [ 3.556548] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.558689] vdc: detected capacity change from 0 to 2621440000 [ 3.593366] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.595852] vdd: detected capacity change from 0 to 2621440000 [ 3.609091] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.611888] vde: detected capacity change from 0 to 4294967296 [ 3.648459] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.650558] vdf: detected capacity change from 0 to 4294967296 [ 3.661693] libphy: Fixed MDIO Bus: probed [ 3.685531] usbcore: registered new interface driver usbserial_generic [ 3.687697] usbserial: USB Serial support registered for generic [ 3.689943] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.694546] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.696160] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.698379] mousedev: PS/2 mouse device common for all mice [ 3.701495] rtc_cmos 00:05: RTC can wake from S4 [ 3.704825] rtc_cmos 00:05: registered as rtc0 [ 3.706313] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.707188] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.710492] intel_pstate: CPU model not supported [ 3.714978] hid: raw HID events driver (C) Jiri Kosina [ 3.718189] usbcore: registered new interface driver usbhid [ 3.719790] usbhid: USB HID core driver [ 3.721559] drop_monitor: Initializing network drop monitor service [ 3.723808] Initializing XFRM netlink socket [ 3.725517] NET: Registered protocol family 10 [ 3.728523] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.728638] Segment Routing with IPv6 [ 3.732811] NET: Registered protocol family 17 [ 3.735398] mpls_gso: MPLS GSO support [ 3.735468] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.739686] RAS: Correctable Errors collector initialized. [ 3.741283] AVX version of gcm_enc/dec engaged. [ 3.742581] AES CTR mode by8 optimization enabled [ 3.818197] sched_clock: Marking stable (3818174102, 0)->(4729521811, -911347709) [ 3.820471] registered taskstats version 1 [ 3.821922] Loading compiled-in X.509 certificates [ 3.823466] zswap: loaded using pool lzo/zbud [ 3.846362] Key type big_key registered [ 3.858450] Key type encrypted registered [ 3.860061] ima: No TPM chip found, activating TPM-bypass! [ 3.861983] ima: Allocated hash algorithm: sha1 [ 3.863305] ima: No architecture policies found [ 3.864853] evm: Initialising EVM extended attributes: [ 3.866445] evm: security.selinux [ 3.867437] evm: security.ima [ 3.868363] evm: security.capability [ 3.869448] evm: HMAC attrs: 0x1 [ 3.871651] rtc_cmos 00:05: setting system clock to 2026-07-04 03:12:45 UTC (1783134765) [ 3.877977] debug: unmapping init [mem 0xffffffffa3603000-0xffffffffa37fffff] [ 3.881639] debug: unmapping init [mem 0xffffffffa2382000-0xffffffffa2658fff] [ 3.892085] Write protecting the kernel read-only data: 28672k [ 3.896070] debug: unmapping init [mem 0xffffffffa0a03000-0xffffffffa0bfffff] [ 3.898586] debug: unmapping init [mem 0xffffffffa1314000-0xffffffffa13fffff] [ 3.935618] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.945045] systemd[1]: Detected virtualization kvm. [ 3.947461] systemd[1]: Detected architecture x86-64. [ 3.949564] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.974216] systemd[1]: No hostname configured. [ 3.975753] systemd[1]: Set hostname to . [ 3.977530] random: systemd: uninitialized urandom read (16 bytes read) [ 3.979777] systemd[1]: Initializing machine ID from random generator. [ 4.047380] random: ln: uninitialized urandom read (6 bytes read) [ 4.151786] random: systemd: uninitialized urandom read (16 bytes read) [ 4.154535] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.165669] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.177130] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... [ OK ] Reached target Initrd Root Device. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. Starting Apply Kernel Variables... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.827383] device-mapper: uevent: version 1.0.3 [ 4.829208] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.542874] virtio_net virtio0 ens2: renamed from eth0 [ 5.593033] random: fast init done [ 5.619437] scsi host0: ata_piix [ 5.647626] scsi host1: ata_piix [ 5.650328] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.653123] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.005769] dracut-initqueue[593]: RTNETLINK answers: File exists [ 10.375948] random: crng init done [ 10.378332] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.776610] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.869559] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.115843] SELinux: Disabled at runtime. [ 12.175580] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.185077] systemd[1]: Detected virtualization kvm. [ 12.187356] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.607931] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.611345] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.615839] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.619245] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.622260] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.629512] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.640559] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Control Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target rpc_pipefs.target. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ 12.717110] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.049905] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.406350] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.419750] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.498135] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.521369] EDAC sbridge: Ver: 1.1.2 [ 15.062452] Key type dns_resolver registered [ 15.394247] NFS: Registering the id_resolver key type [ 15.397872] Key type id_resolver registered [ 15.400285] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Login Service. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Authorization Manager. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg204-server login: [ 58.201727] libcfs: loading out-of-tree module taints kernel. [ 58.240816] Key type ._llcrypt registered [ 58.242470] Key type .llcrypt registered [ 58.329383] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 71.062932] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 72.103387] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 72.114782] alg: No test for adler32 (adler32-zlib) [ 73.257130] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 73.799652] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 75.455178] Key type lgssc registered [ 76.480259] Lustre: Echo OBD driver; http://www.lustre.org/ [ 86.260201] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 113.141583] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 121.217487] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 121.247321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 122.388489] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 122.409895] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 122.496925] Lustre: lustre-MDT0000: new disk, initializing [ 122.609762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 122.630346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 125.080561] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 132.914792] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 132.957829] Lustre: 6508:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 132.981567] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 132.985181] Lustre: Skipped 1 previous similar message [ 133.032410] Lustre: lustre-MDT0001: new disk, initializing [ 133.068697] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 133.090096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 133.101600] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 135.417745] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 138.460035] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 143.655705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 143.780974] Lustre: lustre-OST0000: new disk, initializing [ 143.783413] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 143.786812] Lustre: 8444:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 143.834181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 146.808303] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 147.853881] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 147.869334] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 147.903659] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 155.276271] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 155.354432] Lustre: lustre-OST0001: new disk, initializing [ 155.357932] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 155.362514] Lustre: 9515:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 155.407233] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 157.331990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 157.342306] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 157.383496] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 159.233215] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 168.411708] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 176.654935] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 181.374551] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing check_logdir /tmp/testlogs/ [ 184.290363] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing yml_node [ 187.019917] Lustre: DEBUG MARKER: Client: 2.17.54.83 [ 188.405986] Lustre: DEBUG MARKER: MDS: 2.17.54.83 [ 190.120712] Lustre: DEBUG MARKER: OSS: 2.17.54.83 [ 191.130680] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Jul 3 23:15:51 EDT 2026 [ 201.187532] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 202.117123] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 215.013118] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 215.023988] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 215.043130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 219.105297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 219.105758] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 219.122464] Lustre: Skipped 2 previous similar messages [ 224.226415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 224.232550] Lustre: Skipped 6 previous similar messages [ 229.344096] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 229.347705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 229.359342] Lustre: Skipped 3 previous similar messages [ 229.513730] Lustre: server umount lustre-MDT0000 complete [ 232.021629] LustreError: 6501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783134993 with bad export cookie 5317443257579480489 [ 232.022926] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 232.026472] LustreError: 6501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 234.473324] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 234.477493] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 234.480842] Lustre: Skipped 1 previous similar message [ 234.491873] Lustre: Skipped 1 previous similar message [ 238.271482] Lustre: server umount lustre-MDT0001 complete [ 240.774567] Lustre: server umount lustre-OST0000 complete [ 243.162700] Lustre: server umount lustre-OST0001 complete [ 247.544651] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 254.131688] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 279.942907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 280.094274] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 280.113628] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 280.168435] Lustre: lustre-MDT0000: new disk, initializing [ 280.214697] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 280.226985] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 282.978307] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 290.520545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 290.612370] Lustre: 15066:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 290.669405] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 290.675250] Lustre: Skipped 1 previous similar message [ 290.734376] Lustre: lustre-MDT0001: new disk, initializing [ 290.798128] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 290.834889] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 290.843215] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 293.864880] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 300.531626] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 304.628557] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 313.311395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 313.595822] Lustre: lustre-OST0000: new disk, initializing [ 313.600109] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 313.606661] Lustre: 17033:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 313.696855] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 315.198625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 315.217102] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 315.454845] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 316.780229] hrtimer: interrupt took 4243929 ns [ 320.147502] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 332.622221] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 337.996897] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 338.177869] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 344.708919] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 345.103764] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 351.207238] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 351.209697] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 351.211175] LustreError: Skipped 1 previous similar message [ 351.221792] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 351.221802] Lustre: Skipped 2 previous similar messages [ 351.250904] Lustre: Skipped 1 previous similar message [ 355.588857] Lustre: server umount lustre-OST0000 complete [ 362.464881] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 362.472346] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 369.247262] Lustre: server umount lustre-MDT0000 complete [ 372.199369] LustreError: 15076:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 372.232273] LustreError: 15076:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 373.223746] LustreError: 15077:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 373.253102] LustreError: 15077:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 374.196389] LustreError: 15057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783135135 with bad export cookie 5317443257579482022 [ 374.206628] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 374.209926] LustreError: 15057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 374.464662] Lustre: server umount lustre-MDT0001 complete [ 380.431723] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 23:18:59 (1783135139) [ 388.582193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 389.381601] LustreError: 19300:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 389.485995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 394.235113] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 394.720928] LustreError: 19301:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 399.852186] LustreError: 19300:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 403.480948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 408.310339] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 416.711610] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 424.123127] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 433.536070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 442.413845] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 451.136803] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 455.078878] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 464.354960] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 464.364636] Lustre: Skipped 1 previous similar message [ 464.367757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 464.380764] Lustre: Skipped 6 previous similar messages [ 468.016159] Lustre: server umount lustre-MDT0000 complete [ 469.475150] LustreError: 21415:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 469.497937] LustreError: 21415:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 472.151718] LustreError: 19282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783135233 with bad export cookie 5317443257579482904 [ 472.160168] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 472.173789] LustreError: 19282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 472.511233] Lustre: server umount lustre-MDT0001 complete [ 486.463918] Lustre: server umount lustre-OST0000 complete [ 507.315513] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 517.059065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.413175] LustreError: 23681:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 517.466383] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 517.477389] Lustre: Skipped 2 previous similar messages [ 521.455335] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 529.768657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 534.689251] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 537.560303] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 544.158948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 550.153310] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 558.996532] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 559.161654] Lustre: lustre-OST0001: new disk, initializing [ 559.165643] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 559.170871] Lustre: 25997:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 560.509600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 560.528038] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 560.637550] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 565.920199] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 576.424950] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 580.024388] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 596.978627] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 596.981361] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 597.036501] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 600.356962] Lustre: server umount lustre-MDT0000 complete [ 603.619952] LustreError: 24426:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 603.636781] LustreError: 24426:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 604.358025] LustreError: 23661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783135365 with bad export cookie 5317443257579483744 [ 604.364714] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 604.367917] LustreError: 23661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 604.592526] Lustre: server umount lustre-MDT0001 complete [ 618.919966] Lustre: server umount lustre-OST0000 complete [ 633.449852] Lustre: server umount lustre-OST0001 complete [ 651.654677] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 662.541210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 662.999209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 663.008413] Lustre: Skipped 3 previous similar messages [ 668.142164] LustreError: 29021:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 668.171110] LustreError: 29021:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 668.243664] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 676.894594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 681.811215] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 684.920138] Lustre: 30171:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 691.945115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 699.380757] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 708.482517] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 715.957533] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 724.420762] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 728.082866] Lustre: 32050:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 738.786431] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 738.792188] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 738.810400] Lustre: Skipped 3 previous similar messages [ 738.823705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 738.832773] Lustre: Skipped 4 previous similar messages [ 741.404927] Lustre: server umount lustre-MDT0000 complete [ 745.704056] LustreError: 29002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783135507 with bad export cookie 5317443257579485046 [ 745.719732] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 745.731645] LustreError: 29002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 746.166432] Lustre: server umount lustre-MDT0001 complete [ 761.162903] Lustre: server umount lustre-OST0000 complete [ 775.585759] Lustre: server umount lustre-OST0001 complete [ 786.612454] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 23:25:45 (1783135545) [ 825.257909] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 836.776635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 837.087121] LustreError: 34873:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 837.104715] LustreError: 34873:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 842.282284] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 851.445576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 856.698417] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 860.163885] Lustre: 36020:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 867.518371] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 875.085486] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 885.507830] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 895.606982] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 904.682677] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 908.950435] Lustre: 37905:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 918.501984] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 918.509169] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 918.521930] Lustre: Skipped 3 previous similar messages [ 918.531810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 918.539677] Lustre: Skipped 4 previous similar messages [ 923.734259] Lustre: server umount lustre-MDT0000 complete [ 928.233821] LustreError: 34854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783135689 with bad export cookie 5317443257579486152 [ 928.262580] LustreError: 34854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 928.262854] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 928.893199] Lustre: server umount lustre-MDT0001 complete [ 944.279217] Lustre: server umount lustre-OST0000 complete [ 948.203200] Lustre: 3649:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783135693/real 1783135693] req@ffff982f79b23b80 x1869752393084032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1783135709 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 948.519335] Lustre: server umount lustre-OST0001 complete [ 959.612304] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 23:28:39 (1783135719) [ 979.973244] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 991.851302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 992.549518] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 992.556148] Lustre: Skipped 7 previous similar messages [ 997.217979] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1007.246707] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1014.431585] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1018.576834] Lustre: 41196:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1027.518060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1036.048884] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1045.463671] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1052.839979] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1060.737753] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1064.873713] Lustre: 43076:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1065.865987] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1065.880060] Lustre: Skipped 1 previous similar message [ 1069.537393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1069.555809] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1069.566061] Lustre: Skipped 4 previous similar messages [ 1074.998149] Lustre: server umount lustre-MDT0000 complete [ 1079.881865] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783135841 with bad export cookie 5317443257579487258 [ 1079.888361] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1079.889333] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1086.351105] Lustre: server umount lustre-MDT0001 complete [ 1094.475600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1105.376143] LustreError: 3646:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff982f805fb800 x1869752393137024/t0(0) o250->MGC192.168.202.104@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1105.874094] LustreError: 41559:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1105.907109] LustreError: 41559:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 28 previous similar messages [ 1110.953260] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1115.253364] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1120.242266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1125.175978] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1125.881435] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1125.894723] Lustre: Skipped 1 previous similar message [ 1132.634962] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1134.476934] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1140.581972] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1142.265361] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1151.456126] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1153.620245] Lustre: server umount lustre-MDT0000 complete [ 1157.389812] LustreError: 40026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783135919 with bad export cookie 5317443257579488385 [ 1157.395731] LustreError: 40026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1157.396418] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1157.842281] Lustre: server umount lustre-MDT0001 complete [ 1172.799674] Lustre: server umount lustre-OST0000 complete [ 1188.304285] Lustre: server umount lustre-OST0001 complete [ 1199.504642] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 23:32:39 (1783135959) [ 1215.985567] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1226.401861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1231.592646] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1240.736272] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1245.746304] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1248.733816] Lustre: 48836:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1256.017842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1262.822431] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1272.678843] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1278.296710] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1286.043597] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1290.971733] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1307.618871] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1307.630301] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1307.636313] Lustre: Skipped 20 previous similar messages [ 1311.929053] Lustre: server umount lustre-MDT0000 complete [ 1316.546107] LustreError: 47667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783136078 with bad export cookie 5317443257579489176 [ 1316.563350] LustreError: 47667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1316.579187] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1316.874438] Lustre: server umount lustre-MDT0001 complete [ 1332.557605] Lustre: server umount lustre-OST0000 complete [ 1335.394130] Lustre: 3649:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783136080/real 1783136080] req@ffff982f7fcbb480 x1869752393229056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1783136096 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1335.451068] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1335.471510] Lustre: Skipped 13 previous similar messages [ 1337.083568] Lustre: server umount lustre-OST0001 complete [ 1348.799394] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 23:35:07 (1783136107) [ 1351.686849] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1354.062921] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 23:35:13 (1783136113) [ 1367.492733] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 1379.615404] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1402.238525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1402.566444] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1402.659264] Lustre: lustre-MDT0000: new disk, initializing [ 1402.784296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1407.385790] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1417.263370] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1425.081203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1425.391716] Lustre: lustre-OST0000: new disk, initializing [ 1425.401769] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1425.409837] Lustre: Skipped 1 previous similar message [ 1425.417838] Lustre: 54907:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1427.490044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1427.502499] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1427.548819] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1432.155515] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1442.473827] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1445.194126] Lustre: DEBUG MARKER: create test files [ 1478.624380] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1483.636189] Lustre: server umount lustre-OST0000 complete [ 1487.241697] Lustre: server umount lustre-MDT0000 complete [ 1498.648522] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 1502.378990] Key type lgssc unregistered [ 1502.835777] LNet: 56538:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1502.840547] LNetError: 56538:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1503.854584] LNet: Removed LNI 192.168.202.104@tcp [ 1505.232283] Key type .llcrypt unregistered [ 1505.234146] Key type ._llcrypt unregistered [ 1541.492749] Key type ._llcrypt registered [ 1541.495144] Key type .llcrypt registered [ 1541.628746] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1542.863051] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1542.873102] alg: No test for adler32 (adler32-zlib) [ 1543.988655] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 1544.335404] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 1546.047247] Key type lgssc registered [ 1547.055778] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1553.767425] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1553.790976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1555.175388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1559.485645] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1566.920906] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1573.299995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1573.577349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1579.216371] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1579.692890] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1579.775154] Lustre: 59272:0:(osd_internal.h:1444:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1579.789348] Lustre: 59272:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1579.798614] Lustre: 59272:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1579.806930] Lustre: 59272:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1579.811968] Lustre: 59272:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1579.817873] Lustre: 59272:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1585.558966] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1591.349629] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1598.431508] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1610.721376] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1610.725446] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1610.755564] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1613.978669] Lustre: server umount lustre-OST0000 complete [ 1618.400555] Lustre: server umount lustre-MDT0000 complete [ 1630.166415] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 1633.165418] Key type lgssc unregistered [ 1633.623533] LNet: 60882:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1633.630850] LNetError: 60882:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1633.651420] LNet: Removed LNI 192.168.202.104@tcp [ 1634.510932] Key type .llcrypt unregistered [ 1634.518548] Key type ._llcrypt unregistered [ 1671.677125] Key type ._llcrypt registered [ 1671.678925] Key type .llcrypt registered [ 1671.836789] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1673.527274] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1673.590254] alg: No test for adler32 (adler32-zlib) [ 1674.865895] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 1675.289943] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 1676.943273] Key type lgssc registered [ 1678.194429] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1687.108724] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1687.122386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1688.529887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1692.617892] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1699.537660] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1705.690980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1706.036029] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1713.046377] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1714.224358] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1714.303986] Lustre: 63718:0:(osd_internal.h:1444:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1714.312278] Lustre: 63718:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1714.322017] Lustre: 63718:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1714.336162] Lustre: 63718:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1714.357122] Lustre: 63718:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1714.365497] Lustre: 63718:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1722.052793] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1727.229508] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1739.744953] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1739.748723] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1739.789634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1744.210954] Lustre: server umount lustre-OST0000 complete [ 1748.890392] Lustre: server umount lustre-MDT0000 complete [ 1763.872448] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 1767.510036] Key type lgssc unregistered [ 1767.815563] LNet: 65277:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1767.828722] LNetError: 65277:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1767.869449] LNet: Removed LNI 192.168.202.104@tcp [ 1768.746247] Key type .llcrypt unregistered [ 1768.747954] Key type ._llcrypt unregistered [ 1788.079631] Key type ._llcrypt registered [ 1788.083889] Key type .llcrypt registered [ 1788.239596] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 1807.466986] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1809.313961] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1809.352217] alg: No test for adler32 (adler32-zlib) [ 1810.719120] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 1811.103755] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 1812.911234] Key type lgssc registered [ 1814.650791] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1860.112715] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1860.145629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1861.415182] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1861.449964] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1861.583109] Lustre: lustre-MDT0000: new disk, initializing [ 1861.696380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1861.714903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1866.990735] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1879.519796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1879.822351] Lustre: 68432:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1879.894342] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1879.898170] Lustre: Skipped 1 previous similar message [ 1879.964936] Lustre: lustre-MDT0001: new disk, initializing [ 1880.074878] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1880.120607] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1880.131371] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1886.382757] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1899.503975] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1907.682469] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1915.246919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1915.525042] Lustre: lustre-OST0000: new disk, initializing [ 1915.533259] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1915.547120] Lustre: 70400:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1915.684865] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1922.050786] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1923.653132] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1923.666399] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1923.771916] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1932.187205] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1937.615833] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1937.883312] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1942.182208] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1942.395773] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1944.546570] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1944.546754] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1944.572461] LustreError: Skipped 1 previous similar message [ 1944.575790] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1944.587152] Lustre: Skipped 1 previous similar message [ 1949.671870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1949.685086] Lustre: Skipped 1 previous similar message [ 1954.787940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1954.799368] Lustre: Skipped 1 previous similar message [ 1958.367185] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1958.625181] Lustre: server umount lustre-OST0000 complete [ 1965.030835] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1965.043131] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1965.052867] Lustre: Skipped 2 previous similar messages [ 1965.069607] Lustre: Skipped 1 previous similar message [ 1968.807097] Lustre: server umount lustre-MDT0000 complete [ 1973.152980] LustreError: 68422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783136734 with bad export cookie 3159948825596338468 [ 1973.154193] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1973.168943] LustreError: 68422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1973.474860] Lustre: server umount lustre-MDT0001 complete [ 1983.664702] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 23:45:43 (1783136743) [ 1995.822434] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 2039.002989] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 2051.193409] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 2092.428601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2092.723624] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2092.772448] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2092.859210] Lustre: lustre-MDT0000: new disk, initializing [ 2092.936132] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2092.954494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2098.370608] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2109.640521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2109.812369] Lustre: 76424:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2109.909138] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2109.916950] Lustre: Skipped 1 previous similar message [ 2110.019875] Lustre: lustre-MDT0001: new disk, initializing [ 2110.125803] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2110.172956] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2110.208075] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2114.663466] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2124.746530] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2131.070487] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2137.530104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2137.805261] Lustre: lustre-OST0000: new disk, initializing [ 2137.814582] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2137.820907] Lustre: 78392:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2137.895712] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2144.148324] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2145.328267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2145.351521] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2145.518301] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2154.503511] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2159.236127] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2159.407178] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2164.531771] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2164.690677] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2171.373777] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2171.374361] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2171.376989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2171.419288] Lustre: Skipped 1 previous similar message [ 2172.826787] Lustre: server umount lustre-OST0000 complete [ 2176.993053] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2177.006457] LustreError: Skipped 1 previous similar message [ 2177.012821] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2181.603967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2181.608996] Lustre: Skipped 2 previous similar messages [ 2181.612050] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2191.327148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2191.637408] Lustre: server umount lustre-MDT0000 complete [ 2191.841861] LustreError: 77606:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2191.876154] LustreError: 77606:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2195.681904] LustreError: 76416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783136957 with bad export cookie 3159948825596339350 [ 2195.692178] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2195.695624] LustreError: 76416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2196.128184] Lustre: server umount lustre-MDT0001 complete [ 2205.581262] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 23:49:25 (1783136965) [ 2213.052774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2213.569410] LustreError: 80806:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2213.645722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2218.673751] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2218.977860] LustreError: 80807:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2224.097498] LustreError: 80806:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2227.323068] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2232.193996] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2239.675760] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2245.860544] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2253.182032] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2253.752629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2253.761454] Lustre: Skipped 1 previous similar message [ 2261.950977] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2271.218750] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2275.188650] Lustre: 83477:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting f5c7b9a9-c20b-4b18-b156-746c47945f15 at adminstrative request [ 2278.373516] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2278.394465] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2288.616581] Lustre: MGS: Client f5c7b9a9-c20b-4b18-b156-746c47945f15 (at 0@lo) reconnecting [ 2288.637822] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x2bda64b6fa152808 to 0x2bda64b6fa152aa8 [ 2288.654565] Lustre: MGC192.168.202.104@tcp: Connection restored to 0@lo (at 0@lo) [ 2314.390750] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2314.701130] Lustre: lustre-OST0001: new disk, initializing [ 2314.710309] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2314.732675] Lustre: 83872:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2321.597448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2321.613958] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2321.700642] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2321.998428] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2333.011485] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2337.260615] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2337.272048] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2337.277600] LustreError: Skipped 1 previous similar message [ 2337.285259] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2337.300374] Lustre: Skipped 1 previous similar message [ 2337.331057] Lustre: Skipped 4 previous similar messages [ 2342.631097] Lustre: server umount lustre-OST0001 complete [ 2347.497620] LustreError: 82628:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2347.537589] LustreError: 82628:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2352.938626] Lustre: server umount lustre-OST0000 complete [ 2357.730619] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2357.752755] Lustre: Skipped 3 previous similar messages [ 2363.198867] Lustre: server umount lustre-MDT0000 complete [ 2367.507324] LustreError: 80788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783137129 with bad export cookie 3159948825596340904 [ 2367.512827] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2367.516976] LustreError: 80788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2367.985193] Lustre: server umount lustre-MDT0001 complete [ 2378.520973] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 23:52:17 (1783137137) [ 2380.767505] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2383.123961] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 23:52:22 (1783137142) [ 2385.270825] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2387.322150] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 23:52:27 (1783137147) [ 2389.116888] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2391.367420] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 23:52:31 (1783137151) [ 2393.377861] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2395.480132] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 23:52:35 (1783137155) [ 2397.390410] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2403.783498] loop: module loaded [ 2403.818457] loop0: detected capacity change from 0 to 4294967296 [ 2403.882150] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2406.005737] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2410.128767] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2414.435762] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2442.502121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2442.583630] LustreError: 87999:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2442.600347] LustreError: 87999:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2442.612312] LustreError: 87999:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2442.621179] LustreError: 87999:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2463.996823] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 2479.943917] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 2526.494784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2526.810737] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2526.872309] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2526.991638] Lustre: lustre-MDT0000: new disk, initializing [ 2527.135512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2527.140057] Lustre: Skipped 1 previous similar message [ 2527.163269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2532.430315] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2544.340118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2544.536812] Lustre: 90882:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2544.867679] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2550.389255] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2560.384254] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2568.524927] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2576.557508] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2576.828664] Lustre: 92850:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2583.464678] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2584.155781] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2584.277554] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2592.537294] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2596.994495] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2597.295584] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2601.829320] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2601.989996] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2605.027445] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2605.031251] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2605.044745] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2605.044752] Lustre: Skipped 10 previous similar messages [ 2605.093703] Lustre: Skipped 1 previous similar message [ 2610.447540] Lustre: server umount lustre-OST0000 complete [ 2616.813427] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2622.197391] Lustre: server umount lustre-MDT0000 complete [ 2625.504860] LustreError: 90889:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2625.524825] LustreError: 90889:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2626.803696] LustreError: 90873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783137388 with bad export cookie 3159948825596341520 [ 2626.818196] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2626.822587] LustreError: 90873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2627.423344] Lustre: server umount lustre-MDT0001 complete [ 2638.812740] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 23:56:38 (1783137398) [ 2646.128192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2646.973531] LustreError: 95263:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2651.220499] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2659.258480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2659.656175] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2659.668574] Lustre: Skipped 3 previous similar messages [ 2664.673783] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2673.372279] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2680.085235] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2689.357091] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2697.097563] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2705.624545] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2710.645213] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2710.816926] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2715.869330] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2716.053127] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2722.901972] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2723.063375] Lustre: lustre-OST0001: new disk, initializing [ 2723.066298] Lustre: Skipped 2 previous similar messages [ 2723.068936] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2723.072648] Lustre: Skipped 3 previous similar messages [ 2723.076833] Lustre: 98633:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2729.092828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2729.100329] Lustre: Skipped 2 previous similar messages [ 2729.108774] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2729.256887] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2730.614265] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2740.138407] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2746.536190] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2746.703809] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2752.790553] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2753.112605] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2777.329497] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2803.672965] Lustre: Failing over lustre-MDT0000 [ 2804.055527] Lustre: server umount lustre-MDT0000 complete [ 2806.260674] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2806.267030] LustreError: 95264:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2806.280315] Lustre: Skipped 5 previous similar messages [ 2806.314407] LustreError: 95264:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 2821.608300] Lustre: 66653:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783137567/real 1783137567] req@00000000d45cb88d x1869754214408192/t0(0) o400->MGC192.168.202.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1783137583 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2821.641135] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2829.255828] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2829.261469] LDISKFS-fs (dm-0): recovery complete [ 2829.269822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2831.391156] LustreError: 101452:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2831.403767] LustreError: 101452:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000f8e9e3ec x1869754214413696/t0(0) o250->MGC192.168.202.104@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1783137593 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2831.436585] LustreError: 101452:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2831.860593] LustreError: 66650:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000000df86b5 x1869754214416256/t0(0) o250->MGC192.168.202.104@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2832.264856] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2833.579281] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2833.580080] Lustre: 95259:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2836.696481] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2837.489192] Lustre: 95264:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2837.495074] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2837.503064] Lustre: 95264:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2837.630811] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2837.959814] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2838.316847] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2838.320927] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2838.642596] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2838.651409] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2838.660731] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2838.676315] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2839.329165] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2839.343116] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2839.683894] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2839.692145] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2840.656821] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2840.683042] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 5 previous similar messages [ 2841.336407] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2841.350713] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2841.999128] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2842.006811] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2844.927468] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2844.955500] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2845.631365] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2845.645688] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2846.314658] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2846.325631] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2853.096124] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2853.110418] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 24 previous similar messages [ 2853.760125] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2853.790117] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2854.439172] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2854.447367] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2869.153482] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2869.160707] Lustre: 101482:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 49 previous similar messages [ 2869.804646] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2869.819969] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 49 previous similar messages [ 2870.463786] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2870.474516] LustreError: 101482:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 49 previous similar messages [ 2893.119616] Lustre: 101482:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2893.129499] Lustre: 101482:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2893.141439] Lustre: 101482:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 34cb0843-6dac-44dd-a20d-024c165efdf6@192.168.202.4@tcp [ 2893.164475] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2893.168605] Lustre: 101482:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@00000000efc7802c x1869754194268032/t0(8589934769) o101->34cb0843-6dac-44dd-a20d-024c165efdf6@192.168.202.4@tcp:560/0 lens 592/0 e 0 to 0 dl 1783137665 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2893.228462] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2893.228901] Lustre: lustre-MDT0000: Denying connection for new client 34cb0843-6dac-44dd-a20d-024c165efdf6 (at 192.168.202.4@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2893.288466] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2893.399300] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2893.410241] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2893.413340] Lustre: Skipped 2 previous similar messages [ 2893.522345] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2893.525746] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2914.286711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2914.302134] Lustre: Skipped 6 previous similar messages [ 2927.071178] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2927.202304] Lustre: server umount lustre-OST0000 complete [ 2929.649187] LustreError: 97084:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2929.662279] LustreError: 97084:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 33 previous similar messages [ 2934.245433] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2934.759158] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2934.785064] Lustre: Skipped 3 previous similar messages [ 2937.619495] Lustre: server umount lustre-OST0001 complete [ 2948.014402] Lustre: server umount lustre-MDT0000 complete [ 2952.546013] LustreError: 95244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783137714 with bad export cookie 3159948825596392732 [ 2952.553039] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2952.563821] LustreError: 95244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2953.429415] Lustre: server umount lustre-MDT0001 complete [ 2964.417565] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 00:02:03 (1783137723) [ 2972.583356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2972.603697] Lustre: *** cfs_fail_loc=197, val=0*** [ 2988.321982] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2993.441191] LustreError: 103477:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.104@tcp: failed processing log, type 4: rc = -110 [ 2998.612356] Lustre: *** cfs_fail_loc=197, val=0*** [ 2998.622302] Lustre: Skipped 65 previous similar messages [ 3019.167695] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3019.177029] Lustre: Skipped 3 previous similar messages [ 3030.771692] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3041.627393] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3044.353868] Lustre: server umount lustre-OST0000 complete [ 3056.999233] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 00:03:36 (1783137816) [ 3079.201816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3079.952584] LustreError: 105098:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3079.977442] LustreError: 105098:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3085.827834] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3096.165682] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3100.468621] LustreError: 105129:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3100.480897] LustreError: 105129:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3100.486594] LustreError: 105129:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3100.900902] Lustre: server umount lustre-MDT0000 complete [ 3136.697746] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 3148.877189] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3192.566286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3192.849913] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3192.876567] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3192.972839] Lustre: lustre-MDT0000: new disk, initializing [ 3193.083542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3197.773917] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3208.786450] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3208.925491] Lustre: 109310:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3209.215690] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3213.942569] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3224.572699] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3231.365355] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3240.591859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3240.844841] Lustre: 111277:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3249.779450] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3249.989813] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3261.174920] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3266.456188] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3266.698331] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3272.381058] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3272.673046] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3275.747019] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3275.760173] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3275.766771] LustreError: Skipped 2 previous similar messages [ 3275.769546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3275.769554] Lustre: Skipped 8 previous similar messages [ 3275.803109] Lustre: Skipped 3 previous similar messages [ 3281.217520] Lustre: server umount lustre-OST0000 complete [ 3291.486191] Lustre: server umount lustre-MDT0000 complete [ 3295.531638] LustreError: 112618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783138057 with bad export cookie 3159948825596403309 [ 3295.536236] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3295.547772] LustreError: 112618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3296.086949] Lustre: server umount lustre-MDT0001 complete [ 3306.646487] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 00:07:46 (1783138066) [ 3316.125857] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 3318.840194] Key type lgssc unregistered [ 3319.130969] LNet: 113766:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3319.139824] LNetError: 113766:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3319.159084] LNet: Removed LNI 192.168.202.104@tcp [ 3320.312211] Key type .llcrypt unregistered [ 3320.318308] Key type ._llcrypt unregistered [ 3344.049989] Key type ._llcrypt registered [ 3344.056898] Key type .llcrypt registered [ 3344.419483] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 3366.331953] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3367.081784] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3367.121132] alg: No test for adler32 (adler32-zlib) [ 3368.121723] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 3368.385520] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 3370.047218] Key type lgssc registered [ 3371.372956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3416.014492] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3416.044931] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3417.204933] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3417.225363] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3417.273405] Lustre: lustre-MDT0000: new disk, initializing [ 3417.331692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3417.344799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3422.426821] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3436.636941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3436.994204] Lustre: lustre-OST0000: new disk, initializing [ 3437.000578] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3437.007540] Lustre: Skipped 1 previous similar message [ 3437.023261] Lustre: 117942:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3437.145079] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3444.877914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3444.897643] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3444.934454] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3445.366250] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3456.868919] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3457.113855] Lustre: lustre-OST0001: new disk, initializing [ 3457.117536] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3457.123405] Lustre: 118805:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3457.193603] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3463.708984] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3463.860368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3463.872206] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3463.908144] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3473.891315] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3473.919693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3473.926256] Lustre: Skipped 1 previous similar message [ 3479.009116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3479.019348] Lustre: Skipped 1 previous similar message [ 3479.572843] Lustre: server umount lustre-MDT0000 complete [ 3485.965586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3490.525867] LustreError: 119329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783138252 with bad export cookie 14672220782804050726 [ 3490.541549] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3490.878573] Lustre: server umount lustre-OST0000 complete [ 3494.895807] Lustre: server umount lustre-OST0001 complete [ 3503.761553] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 00:11:03 (1783138263) [ 3514.466687] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3540.576772] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 00:11:39 (1783138299) [ 3583.334573] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 3599.807721] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3644.753901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3645.065505] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3645.091772] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3645.208613] Lustre: lustre-MDT0000: new disk, initializing [ 3645.363131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3645.382098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3651.127524] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3663.553953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3663.694985] Lustre: 125576:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3663.765683] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3663.769218] Lustre: Skipped 1 previous similar message [ 3663.846353] Lustre: lustre-MDT0001: new disk, initializing [ 3663.928428] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3663.983484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3663.996504] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3668.948606] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3679.604535] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3687.023991] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3695.534885] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3695.897989] Lustre: lustre-OST0000: new disk, initializing [ 3695.901836] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3695.909439] Lustre: 127545:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3696.017627] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3702.341354] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3704.945039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3704.965323] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3705.217219] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3714.601071] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3721.482204] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3721.773640] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3727.241977] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3727.470410] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3730.915125] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3730.921370] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3730.925665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3730.931656] Lustre: Skipped 2 previous similar messages [ 3730.943368] LustreError: Skipped 1 previous similar message [ 3735.856426] Lustre: server umount lustre-OST0000 complete [ 3740.640648] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3740.648573] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3740.662809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3740.666536] Lustre: Skipped 1 previous similar message [ 3745.794184] Lustre: server umount lustre-MDT0000 complete [ 3749.872380] LustreError: 127544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783138511 with bad export cookie 14672220782804051944 [ 3749.874301] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3749.880623] LustreError: 127544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3750.301307] Lustre: server umount lustre-MDT0001 complete [ 3760.862827] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 00:15:20 (1783138520) [ 3769.140788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3769.505566] LustreError: 129961:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3769.521231] LustreError: 129961:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3769.574415] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3774.951203] LustreError: 129962:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3775.064552] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3780.068412] LustreError: 129961:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3785.186879] LustreError: 129962:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3785.410656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3790.439869] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3798.453827] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3804.413312] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3812.166808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3812.454314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3812.462964] Lustre: Skipped 1 previous similar message [ 3819.000216] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3827.093902] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3836.780433] Lustre: *** cfs_fail_loc=159, val=0*** [ 3853.027459] Lustre: lustre-MDT0000: Client 41bb12da-144c-4f16-b73d-e57b78be7715 (at 192.168.202.4@tcp) reconnecting [ 3854.849229] Lustre: *** cfs_fail_loc=159, val=0*** [ 3854.852330] Lustre: Skipped 5 previous similar messages [ 3871.478751] Lustre: lustre-MDT0000: Client 41bb12da-144c-4f16-b73d-e57b78be7715 (at 192.168.202.4@tcp) reconnecting [ 3891.689082] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3891.709240] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3891.737366] Lustre: Skipped 1 previous similar message [ 3891.744353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3891.763125] Lustre: Skipped 2 previous similar messages [ 3895.317307] Lustre: server umount lustre-OST0000 complete [ 3902.962701] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3902.970874] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3902.978422] Lustre: Skipped 2 previous similar messages [ 3902.985404] Lustre: Skipped 3 previous similar messages [ 3906.373849] Lustre: server umount lustre-MDT0000 complete [ 3908.068760] LustreError: 132681:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3908.092440] LustreError: 132681:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3911.546230] LustreError: 129941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783138673 with bad export cookie 14672220782804052833 [ 3911.555898] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3911.560229] LustreError: 129941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3912.101609] Lustre: server umount lustre-MDT0001 complete [ 3924.767861] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 3927.937490] Key type lgssc unregistered [ 3928.194965] LNet: 133736:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3928.204823] LNetError: 133736:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3928.223262] LNet: Removed LNI 192.168.202.104@tcp [ 3929.084610] Key type .llcrypt unregistered [ 3929.091598] Key type ._llcrypt unregistered [ 3957.520985] Key type ._llcrypt registered [ 3957.524677] Key type .llcrypt registered [ 3957.637848] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3959.026461] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3959.334803] alg: No test for adler32 (adler32-zlib) [ 3960.466372] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 3960.843704] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 3962.560414] Key type lgssc registered [ 3964.210792] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3972.559871] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3972.603266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3974.268686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3979.869857] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3983.848257] LustreError: 135153:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3983.877857] LustreError: 135153:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3988.973672] LustreError: 135154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3990.814733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3991.074770] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3997.114492] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4005.904496] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4012.460438] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4019.647292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4020.066851] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4026.553891] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4034.609617] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4043.656715] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4054.442662] Lustre: *** cfs_fail_loc=159, val=0*** [ 4070.107030] Lustre: lustre-MDT0000: Client f28efad6-e168-4aa3-83e5-6a94971b2d3c (at 192.168.202.4@tcp) reconnecting [ 4072.796308] Lustre: *** cfs_fail_loc=159, val=0*** [ 4089.037760] Lustre: lustre-MDT0001: Client f28efad6-e168-4aa3-83e5-6a94971b2d3c (at 192.168.202.4@tcp) reconnecting [ 4090.803852] Lustre: *** cfs_fail_loc=159, val=0*** [ 4090.810774] Lustre: Skipped 3 previous similar messages [ 4107.524626] Lustre: lustre-MDT0001: Client f28efad6-e168-4aa3-83e5-6a94971b2d3c (at 192.168.202.4@tcp) reconnecting [ 4112.346874] Lustre: 138218:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4115.052976] Lustre: *** cfs_fail_loc=159, val=0*** [ 4115.058470] Lustre: Skipped 4 previous similar messages [ 4131.567521] Lustre: lustre-MDT0000: Client 97fbfb96-bf62-4384-9c4b-2ba69bc3bc22 (at 192.168.202.4@tcp) reconnecting [ 4133.440638] Lustre: *** cfs_fail_loc=159, val=0*** [ 4133.445104] Lustre: Skipped 14 previous similar messages [ 4148.966414] Lustre: lustre-MDT0000: Client 97fbfb96-bf62-4384-9c4b-2ba69bc3bc22 (at 192.168.202.4@tcp) reconnecting [ 4150.698530] Lustre: 138507:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4158.443082] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4158.452822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4158.478446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4159.983196] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4159.990679] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4160.003677] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4161.524533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4161.534195] Lustre: Skipped 1 previous similar message [ 4163.185346] Lustre: server umount lustre-OST0000 complete [ 4169.531713] Lustre: server umount lustre-MDT0000 complete [ 4170.208450] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4170.218537] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4170.238678] LustreError: 135153:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4173.426417] LustreError: 135133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783138935 with bad export cookie 10882558923378957458 [ 4173.437276] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4173.447117] LustreError: 135133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4173.815910] Lustre: server umount lustre-MDT0001 complete [ 4185.456776] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4188.052983] Key type lgssc unregistered [ 4188.381862] LNet: 139560:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4188.388954] LNetError: 139560:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4188.411822] LNet: Removed LNI 192.168.202.104@tcp [ 4189.468166] Key type .llcrypt unregistered [ 4189.471954] Key type ._llcrypt unregistered [ 4214.608633] Key type ._llcrypt registered [ 4214.613774] Key type .llcrypt registered [ 4214.725806] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4215.552914] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4215.703922] alg: No test for adler32 (adler32-zlib) [ 4216.795852] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 4217.096840] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4218.783357] Key type lgssc registered [ 4219.851819] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4226.539592] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4226.572175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4227.960539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4231.684728] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4238.307570] LustreError: 140977:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4238.339017] LustreError: 140977:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4238.978388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4239.230912] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4242.194379] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4248.856270] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4254.567884] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4260.318239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4260.533447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4263.626718] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4265.927702] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4272.846536] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4273.689715] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4281.144440] Lustre: 143566:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4285.444287] Lustre: 143615:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4287.697522] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4289.268290] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4294.595200] Lustre: 143763:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4295.606717] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4296.606056] Lustre: 143862:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4303.841652] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4303.847090] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4303.863663] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4303.899699] Lustre: Skipped 1 previous similar message [ 4308.123939] Lustre: server umount lustre-OST0000 complete [ 4314.081110] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4314.099886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4314.104815] Lustre: Skipped 2 previous similar messages [ 4319.200375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4324.321739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4324.338142] Lustre: Skipped 2 previous similar messages [ 4326.367322] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4326.575671] Lustre: server umount lustre-MDT0000 complete [ 4329.441063] LustreError: 141704:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4329.470866] LustreError: 141704:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4330.008213] LustreError: 140957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783139091 with bad export cookie 9491845734218087506 [ 4330.015682] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4330.027616] LustreError: 140957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4330.385535] Lustre: server umount lustre-MDT0001 complete [ 4339.239892] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4341.309712] Key type lgssc unregistered [ 4341.523694] LNet: 144924:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4341.528382] LNetError: 144924:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4341.540935] LNet: Removed LNI 192.168.202.104@tcp [ 4342.146309] Key type .llcrypt unregistered [ 4342.148745] Key type ._llcrypt unregistered [ 4365.351452] Key type ._llcrypt registered [ 4365.352736] Key type .llcrypt registered [ 4365.421760] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4366.359322] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4366.482992] alg: No test for adler32 (adler32-zlib) [ 4367.526391] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 4367.774721] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4369.439192] Key type lgssc registered [ 4370.744435] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4379.340421] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4379.387862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4381.251590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4386.775959] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4391.395887] LustreError: 146339:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4391.418916] LustreError: 146339:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4396.137503] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4396.677702] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4400.797473] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4408.337337] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4415.230436] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4423.667823] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4424.223856] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4427.326585] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4432.032706] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4439.713673] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4441.677026] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4446.411973] Lustre: *** cfs_fail_loc=159, val=0*** [ 4461.805515] Lustre: lustre-MDT0000: Client 003e35ba-8f22-48d5-9c36-440260b8e77d (at 192.168.202.4@tcp) reconnecting [ 4482.530683] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4482.537326] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4482.560335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4486.186948] Lustre: server umount lustre-OST0000 complete [ 4492.769862] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4492.798584] Lustre: Skipped 1 previous similar message [ 4492.814507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4492.817281] Lustre: Skipped 1 previous similar message [ 4496.992527] Lustre: server umount lustre-MDT0000 complete [ 4497.888899] LustreError: 148889:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4497.910630] LustreError: 148889:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4500.864263] LustreError: 146320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783139262 with bad export cookie 18027010579537215781 [ 4500.865889] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4500.873109] LustreError: 146320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4501.124097] Lustre: server umount lustre-MDT0001 complete [ 4514.229029] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4517.255143] Key type lgssc unregistered [ 4517.661499] LNet: 150006:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4517.682642] LNetError: 150006:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4517.702844] LNet: Removed LNI 192.168.202.104@tcp [ 4518.906651] Key type .llcrypt unregistered [ 4518.908462] Key type ._llcrypt unregistered [ 4548.170478] Key type ._llcrypt registered [ 4548.172521] Key type .llcrypt registered [ 4548.285913] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4549.500610] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4549.663830] alg: No test for adler32 (adler32-zlib) [ 4550.860542] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 4551.040344] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4552.730045] Key type lgssc registered [ 4554.041806] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4561.520068] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4561.546576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4563.027311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4567.910267] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4573.157158] LustreError: 151420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4573.178644] LustreError: 151420:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4577.907102] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4578.299736] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4583.772724] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4593.176902] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4599.505763] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4607.509211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4607.816375] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4608.886830] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4614.410468] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4622.241835] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4623.247422] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4642.832346] Lustre: lustre-OST0000: evicting 2233381c-de81-4aec-9b0b-63185ed64e9a (at 192.168.202.4@tcp) by administrative request [ 4642.850768] Lustre: Skipped 1 previous similar message [ 4674.531093] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4674.537222] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4674.544384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4674.560612] Lustre: Skipped 1 previous similar message [ 4679.661229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4679.678676] Lustre: Skipped 1 previous similar message [ 4684.779033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4684.800746] Lustre: Skipped 1 previous similar message [ 4686.303682] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4686.535982] Lustre: server umount lustre-OST0000 complete [ 4695.008889] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4695.015071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4695.029535] Lustre: Skipped 1 previous similar message [ 4695.044248] Lustre: Skipped 2 previous similar messages [ 4697.645760] Lustre: server umount lustre-MDT0000 complete [ 4700.128908] LustreError: 151420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4700.141356] LustreError: 151420:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4701.025640] LustreError: 151400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783139462 with bad export cookie 15318427450103230246 [ 4701.040196] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4701.044597] LustreError: 151400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4701.456367] Lustre: server umount lustre-MDT0001 complete [ 4713.070291] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4716.120864] Key type lgssc unregistered [ 4716.401806] LNet: 155489:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4716.412642] LNetError: 155489:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4716.435593] LNet: Removed LNI 192.168.202.104@tcp [ 4717.567346] Key type .llcrypt unregistered [ 4717.576780] Key type ._llcrypt unregistered [ 4801.041758] Key type ._llcrypt registered [ 4801.044608] Key type .llcrypt registered [ 4801.132944] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4802.187029] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4802.388332] alg: No test for adler32 (adler32-zlib) [ 4803.581087] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 4804.011117] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4805.672660] Key type lgssc registered [ 4807.268176] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4815.409650] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4815.441424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4817.000386] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4822.267418] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4827.105817] LustreError: 157982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4827.114354] LustreError: 157982:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4831.846893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4832.340489] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4836.444883] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4845.943943] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4851.919401] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4859.516624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4860.138042] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4862.236141] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4866.661361] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4874.953632] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4876.532842] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4887.009921] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4887.028796] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4887.039420] Lustre: Skipped 1 previous similar message [ 4887.050259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4887.057979] Lustre: Skipped 1 previous similar message [ 4891.472454] Lustre: server umount lustre-OST0000 complete [ 4897.250164] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4897.256956] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4897.283033] Lustre: Skipped 1 previous similar message [ 4897.309144] Lustre: Skipped 1 previous similar message [ 4902.377898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4902.868939] Lustre: server umount lustre-MDT0000 complete [ 4906.465100] LustreError: 157978:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4906.482846] LustreError: 157978:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4907.400200] LustreError: 158721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783139669 with bad export cookie 6636761886351346982 [ 4907.408432] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4907.419972] LustreError: 158721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4907.899301] Lustre: server umount lustre-MDT0001 complete [ 4919.814164] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4922.377432] Key type lgssc unregistered [ 4922.627651] LNet: 161523:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4922.636166] LNetError: 161523:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4923.688853] LNet: Removed LNI 192.168.202.104@tcp [ 4924.543119] Key type .llcrypt unregistered [ 4924.548339] Key type ._llcrypt unregistered [ 4951.004709] Key type ._llcrypt registered [ 4951.006804] Key type .llcrypt registered [ 4951.091965] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4952.564481] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4952.679602] alg: No test for adler32 (adler32-zlib) [ 4954.016557] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 4954.453511] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4956.321031] Key type lgssc registered [ 4957.914751] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4979.791604] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 00:35:39 (1783139739) [ 4997.367883] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 5008.691126] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 5044.467444] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5044.505306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5045.807585] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5045.865160] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5045.955830] Lustre: lustre-MDT0000: new disk, initializing [ 5046.045209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5046.065625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5050.641089] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5061.845310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5061.961595] Lustre: 165789:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5062.010709] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5062.017580] Lustre: Skipped 1 previous similar message [ 5062.113608] Lustre: lustre-MDT0001: new disk, initializing [ 5062.231329] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5062.279727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5062.289815] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5066.501796] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5075.287587] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5080.606429] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5087.851252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5088.131482] Lustre: lustre-OST0000: new disk, initializing [ 5088.135702] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5088.147546] Lustre: 167759:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5088.330390] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5093.519680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5093.546544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5093.680644] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5096.829102] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5108.379457] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5124.577113] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5124.584285] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5124.591371] Lustre: Skipped 1 previous similar message [ 5124.593689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5124.607026] Lustre: Skipped 1 previous similar message [ 5126.429170] Lustre: server umount lustre-OST0000 complete [ 5133.800197] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5133.810159] LustreError: Skipped 1 previous similar message [ 5133.816794] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5133.827831] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5134.816813] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5134.835081] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5134.843464] Lustre: Skipped 1 previous similar message [ 5137.492799] Lustre: server umount lustre-MDT0000 complete [ 5142.514017] LustreError: 165784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783139904 with bad export cookie 86205498552493682 [ 5142.522843] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5142.557800] LustreError: 165784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5143.225447] Lustre: server umount lustre-MDT0001 complete [ 5159.413793] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 5162.113899] Key type lgssc unregistered [ 5162.398615] LNet: 169663:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5162.407811] LNetError: 169663:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5162.419888] LNet: Removed LNI 192.168.202.104@tcp [ 5163.355200] Key type .llcrypt unregistered [ 5163.357539] Key type ._llcrypt unregistered [ 5190.387415] Key type ._llcrypt registered [ 5190.390391] Key type .llcrypt registered [ 5190.504851] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 5191.882394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5191.927288] alg: No test for adler32 (adler32-zlib) [ 5192.999286] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 5193.360542] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 5195.087392] Key type lgssc registered [ 5196.650981] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5205.125691] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5205.155769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5206.712800] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5212.375397] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5217.258357] LustreError: 171077:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5217.316310] LustreError: 171077:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5222.370511] LustreError: 171078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5223.077136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5223.318318] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5228.028548] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5236.654385] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5243.242151] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5250.818251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5251.213933] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5252.296172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5256.909691] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5264.926373] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5274.735208] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5274.775249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5274.790107] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5274.800114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.820297] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5276.198905] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5276.246316] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5276.271138] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5276.282973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.302556] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5277.198159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.151724] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5278.160221] Lustre: Skipped 1 previous similar message [ 5278.180137] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5278.190338] Lustre: Skipped 1 previous similar message [ 5278.199250] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5278.204626] Lustre: Skipped 1 previous similar message [ 5278.207795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.214602] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5278.228236] Lustre: Skipped 1 previous similar message [ 5279.204168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.991507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.884936] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5280.891551] Lustre: Skipped 2 previous similar messages [ 5280.904792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5280.913345] Lustre: Skipped 2 previous similar messages [ 5280.919630] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5280.923432] Lustre: Skipped 2 previous similar messages [ 5280.927083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.938165] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5280.945409] Lustre: Skipped 2 previous similar messages [ 5281.862570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.761793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.596660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.532612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.338781] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5285.343160] Lustre: Skipped 4 previous similar messages [ 5285.359332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5285.368447] Lustre: Skipped 4 previous similar messages [ 5285.372090] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5285.377538] Lustre: Skipped 4 previous similar messages [ 5285.380768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.397665] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5285.406901] Lustre: Skipped 4 previous similar messages [ 5286.267501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.135996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.000312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.961459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.899519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.752252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.611252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.419138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.317985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.064570] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5294.070649] Lustre: Skipped 9 previous similar messages [ 5294.101260] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5294.114040] Lustre: Skipped 9 previous similar messages [ 5294.117861] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5294.124029] Lustre: Skipped 9 previous similar messages [ 5294.129480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.138228] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5294.146425] Lustre: Skipped 9 previous similar messages [ 5294.947423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.701862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.637844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.461548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.599580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.610365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.685666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.509154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.490479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.278940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.418841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.403928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.326076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.276294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.278274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.273416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.136931] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5310.141887] Lustre: Skipped 16 previous similar messages [ 5310.157772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5310.165919] Lustre: Skipped 16 previous similar messages [ 5310.171951] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5310.178335] Lustre: Skipped 16 previous similar messages [ 5310.183332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.194602] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5310.200808] Lustre: Skipped 16 previous similar messages [ 5311.096610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.223838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.142820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.212648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.216659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.354946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.130445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.931445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.769181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.482609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.218115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.308745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.259810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.435784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.379418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.324675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.045431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.002223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.083087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.003429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.063410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.781603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.718971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.356394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.895023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.664876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.568617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.402116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.303943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.012139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.888237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.706721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.746531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.827815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.836390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.771620] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5342.784633] Lustre: Skipped 35 previous similar messages [ 5342.827881] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5342.846321] Lustre: Skipped 35 previous similar messages [ 5342.856880] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5342.869445] Lustre: Skipped 35 previous similar messages [ 5342.897453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.914650] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5342.931743] Lustre: Skipped 35 previous similar messages [ 5343.829174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.675530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.477232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.567146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.485724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.385681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.304601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.320744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.023688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.052563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.896689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.028923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.074081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.295039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.073864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.818212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.691824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.740815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.816175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.518778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.619280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.635906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.365534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.298167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.173552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.192810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.249582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.661987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.933462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.153131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.829330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.963039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.351708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.063306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.536623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.553809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.080249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.667330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.093161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.402436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.646333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.410337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.608885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.287859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.105142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.873763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.693787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.181758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.603363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.477545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.559042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.478305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.389954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.424559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.739014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.891460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.808248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.118467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.396449] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5407.405678] Lustre: Skipped 58 previous similar messages [ 5407.423702] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5407.437465] Lustre: Skipped 58 previous similar messages [ 5407.446552] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5407.452953] Lustre: Skipped 58 previous similar messages [ 5407.458136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.480754] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5407.510052] Lustre: Skipped 58 previous similar messages [ 5408.697165] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5408.708176] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5408.714977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.543462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.133280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.635080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.473603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.181175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.925778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.084950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.078196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.058458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.892493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.096848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.894827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.880210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.843219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.812978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.867974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.018840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.091275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.805615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.813881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.726252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.648502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.543343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.417557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.471902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.768623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.094832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.071919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.984191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.850445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.872532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.532069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.395460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.246641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.965617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.608923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.269427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.178320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.910841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.638526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.604320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.399217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.298767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.041196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.777164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.785200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.803282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.498321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.375633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.126654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.036134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.841444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.750921] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 5458.761954] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 5458.785735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.842490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.791258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.432259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.012162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.876395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.720176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.681191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.635227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.660920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.667797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.630944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.782695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.098725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.375611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.486294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.382946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.340443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.280436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.126882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.038287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.912980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.633658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.729449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.586325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.540905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.388102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.256278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.126111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.045231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.924185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.868502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.666506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.852808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.995863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.915511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.956345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.976532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.111099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.219668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.104059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.199168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.965411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.984099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.827430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.844767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.113044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.237076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.229286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.105713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.053953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.071918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.999499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.873596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.626441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.625150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.395949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.264177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.448486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.415673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.496569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.365111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.227810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.243224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.153281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.033910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.946472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.692929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.559521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.496443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.479128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.296454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.152047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.148922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.901512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.970416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.040433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.073263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.926380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.930782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.936916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.147813] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5536.159714] Lustre: Skipped 134 previous similar messages [ 5536.176451] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5536.197491] Lustre: Skipped 134 previous similar messages [ 5536.202669] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5536.210431] Lustre: Skipped 134 previous similar messages [ 5536.214928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.228981] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5536.232110] Lustre: Skipped 134 previous similar messages [ 5537.160207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.302897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.186981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.144961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.055600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.138618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.005890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.954465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.981868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.781630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.677654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.567574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.547819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.466526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.311077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.999260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.815201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.560664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.451506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.215979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.204353] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5555.215091] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5555.221835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.299126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.401455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.242894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.179291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.085314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.970649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.691578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.774455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.582555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.565651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.637725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.743976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.021065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.940685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.936657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.014707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.205135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.391668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.426690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.476634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.774044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.690170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.730937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.768815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.721209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.756733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.562073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.465104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.228723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.293830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.183713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.250547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.140139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.094677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.842266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.824687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.576509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.544649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.398325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.390879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.454888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.247443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.330516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.548074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.478936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.713932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.836175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.946814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.284270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.398099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.731061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.756847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.061254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.991567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.134548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.097085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.265670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.165319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.450701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.734280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.235732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.277298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.357871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.195707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.289355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.175772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.159232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.300768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.444772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.436892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.729546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.953020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.867852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.764462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.677622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.636862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.442777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.555148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.467406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.384630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.316659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.259773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.280949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.067963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.928701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.685846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.530498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.399789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.309921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.318762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.276532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.255592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.408758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.633060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.515392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.250933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.226404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.136207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.353747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.544135] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5656.555869] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5656.587774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.696354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.614918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.360062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.196208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.240576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.109804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.227336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.060450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.807656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.644686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.493487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.406743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.286251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.020184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.733151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.502634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.369359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.138674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.956195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.727634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.500733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.343589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.350871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.317564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.420204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.203754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.130217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.920655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.935591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.033940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.978433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.735142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.467470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.277148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.969437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.770218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.374969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.097767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.757499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.623474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.461199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.137183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.841690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.652534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.276861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.128811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.808635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.558541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.253179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.077519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.838338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.602724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.328265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.098855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.975381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.948150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.762655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.570927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.370407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.567674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.470285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.363511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.337984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.600926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.758890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.743050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.825732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.970974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.788099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.929508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.857354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.756951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.560737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.345818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.112265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.028069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.051712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.822518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.708687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.580977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.456246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.509976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.356760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.271714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.053489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.098780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.931092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.993606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.928188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.934526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.810320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.868099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.578316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.393645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.382160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.205393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.979306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.816732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.579220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.509838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.207887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.031520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.786949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.700628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.528624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.591413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.416349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.365537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.195875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.301397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.115116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.078414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.918853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.835549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.797917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.653660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.672823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.767657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.006740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.118129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.327964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.308629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.346591] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5765.363816] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5765.381743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.033066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.736938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.542315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.623471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.531349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.800857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.699564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.819921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.687807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.877903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.886473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.771332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.517068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.386972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.121334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.043972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.933596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.763511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.644388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.461633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.508042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.498170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.440817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.107762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.943945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.660800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.492352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.210485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.001596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.684342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.470669] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5792.484142] Lustre: Skipped 274 previous similar messages [ 5792.499557] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5792.506647] Lustre: Skipped 274 previous similar messages [ 5792.510280] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5792.514836] Lustre: Skipped 274 previous similar messages [ 5792.518858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.539806] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5792.546173] Lustre: Skipped 274 previous similar messages [ 5793.252865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.077652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.835267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.539485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.345501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.117422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.940368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.760417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.452373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.160286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.851194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.616639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.274269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.141160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.949070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.713527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.483676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.161786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.784756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.528744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.225398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.106841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.783729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.623706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.560702] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5811.567802] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5811.583877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.253584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.020910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.767111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.520345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.247118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.244432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.920353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.712294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.641983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.381025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.176709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.955458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.652546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.471464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.291605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.229826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.025360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.875342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.757176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.604869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.388295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.266499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.211909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.193057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.329413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.230710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.107655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.907394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.689482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.382838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.128467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.865785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.836173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.530489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.437469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.274403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.241942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.153295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.139041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.052965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.029553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.813536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.586978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.348566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.138664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.968737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.749222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.567739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.331565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.179828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.996166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.902702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.842778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.518313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.281545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.025906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.855863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.720785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.555929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.493646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.355414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.010401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.863024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.625040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.387860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.181511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.070805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.942296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.972401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.730391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.612614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.425123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.263868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.885287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.541489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.273747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.113797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.789242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.587144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.272924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.104683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.829712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.502503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.230601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.915437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.606680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.404415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.033567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.843035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.428232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.150351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.037846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.931646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.796790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.602788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.365627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.140175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.052699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.845670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.649915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.465862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.304530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.034769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.763688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.632129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.377294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.138451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.001334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.829940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.560439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.451715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.211921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.867937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.573953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.538152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.225695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.856418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.612558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.270263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.879464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.685450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.419537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.327744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.183200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.013715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.706600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.518050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.257078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.125445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.017653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.966619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.713149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.571671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.405755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.426238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.306916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.287580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.059107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.824446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.606565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.427727] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5925.435269] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5925.440809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.302706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.114209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.818488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.461641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.191358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.963287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.676995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.449591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.165627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.027145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.769747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.757025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.738635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.762829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.928288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.995687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.275554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.306180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.467462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.644217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.536578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.454062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.337960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.168628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.824879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.609292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.324871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.029945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.795368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.805536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.696539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.589806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.440641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.222931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.150155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.897247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.718795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.513433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.371985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.930985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.796652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.828209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.666629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.547304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.574671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.339075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.079946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.891447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.638188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.439670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.372466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.232102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.059508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.986633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.904244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.744725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.644870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.443098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.264591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.004150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.732644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.411523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.257526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.928676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.779463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.520934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.332791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.114262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.903146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.648318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.357516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.107246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.950867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.771635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.527639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.371497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.129158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.934753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.713521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.491937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.395493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.040237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.771910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.488065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.370265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.115439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.944524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.625487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.511457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.437211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.338494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.132642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.003265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.989567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.861665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.645978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.524438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.175122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.862621] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6007.868314] LustreError: 171097:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6007.873952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.465809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.193335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.995065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.693119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.302731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.109242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.983525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.793099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.523638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.453591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.278428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.054257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.962864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.919586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.858980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.049857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.969944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.929981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.048248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.003359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.206666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.029136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.871983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.623524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.448030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.113135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.798823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.434176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.121818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.848119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.616727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.479584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.178440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.869653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.529242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.186662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.950204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.549914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.196967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.894576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.678955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.345284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.101522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.788271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.497373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.204792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.113522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.950804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.891904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.716473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.514046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.473220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.284345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.125756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.854727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.578593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.396832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.196319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.983624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.885365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.711623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.573806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.331502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.120139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.915195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.755733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.435485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.201416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.045529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.981415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.789667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.664150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.528615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.452906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.513939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.536480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.620141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.761110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.091800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.954550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.994608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.841845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.750199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.522743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.327760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.229287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.104690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.949722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.940365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.999123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.895492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.883903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.794886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.818646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.589583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.594540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.419977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.224651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.127544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.088392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.978856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.073459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.132880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.387268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.645722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.982370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.086396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.068328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.095791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.920063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.718317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.630962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.451718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.235242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.143367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.928307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.683794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.522532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.364662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.111700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.997816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.715897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.653211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.483384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.334277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.075246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.927098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.787680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.567678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.485139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.458747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.472491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.297429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.116422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.017642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.919791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.716872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.516662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.326695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.203607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.056835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.909135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.790261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.840552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.721239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.579969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.580592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.454111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.380780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.287981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.091710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.003126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.905860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.769286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.636954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.642303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.552084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.573791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.501794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.342242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.193437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.345295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.303163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.237705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.115789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.977947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.736674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.515905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.323599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.224153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.143953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.026072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.852594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.651960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.644932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.562071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.628285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.830549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.721824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.389480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.087708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.750514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.512731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.230243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.989744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.582370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.283509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.918483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.698839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.539866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.395411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.277478] LustreError: 172900:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6480.282705] LustreError: 171097:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6819.311299] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6819.322352] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6819.338182] Lustre: Skipped 456 previous similar messages [ 6819.343706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6822.369029] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6822.370751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6822.385133] Lustre: Skipped 1 previous similar message [ 6823.828159] Lustre: server umount lustre-OST0000 complete [ 6827.494275] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6827.502523] Lustre: Skipped 1 previous similar message [ 6832.610595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6832.615658] Lustre: Skipped 1 previous similar message [ 6833.887291] Lustre: server umount lustre-MDT0000 complete [ 6837.739261] LustreError: 172982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6837.760532] LustreError: 172982:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6837.806063] LustreError: 171058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783141599 with bad export cookie 16571522253529660516 [ 6837.808798] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6837.814321] LustreError: 171058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6838.349660] Lustre: server umount lustre-MDT0001 complete [ 6848.719604] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 6851.072512] Key type lgssc unregistered [ 6851.290781] LNet: 226274:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6851.299920] LNetError: 226274:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6851.323815] LNet: Removed LNI 192.168.202.104@tcp [ 6852.142690] Key type .llcrypt unregistered [ 6852.145184] Key type ._llcrypt unregistered [ 6876.325567] Key type ._llcrypt registered [ 6876.327558] Key type .llcrypt registered [ 6876.422959] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 6877.775791] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6877.798942] alg: No test for adler32 (adler32-zlib) [ 6878.886790] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 6879.117351] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 6880.791361] Key type lgssc registered [ 6881.868229] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6889.339882] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6889.352240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6890.720938] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6894.863949] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 6901.233342] LustreError: 227690:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6901.261993] LustreError: 227690:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6905.556056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6906.071144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6911.084411] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 6918.831250] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6924.527478] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6931.546891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6931.955254] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6936.059741] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6940.018064] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 6946.174462] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6955.609932] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6957.610816] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6957.619894] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6957.625892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.638618] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6957.639539] LustreError: 227710:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6959.800604] Lustre: lustre-OST0000: Not available for connect from 192.168.202.4@tcp (stopping) [ 6961.633546] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6961.634360] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6961.658492] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6961.670832] Lustre: Skipped 1 previous similar message [ 6961.683791] Lustre: Skipped 1 previous similar message [ 6964.890031] Lustre: lustre-OST0000: Not available for connect from 192.168.202.4@tcp (stopping) [ 6965.751049] Lustre: server umount lustre-OST0000 complete [ 6977.679586] LustreError: 227710:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7000.537276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7000.963574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7002.036288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.055660] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.057919] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7002.081435] Lustre: Skipped 1 previous similar message [ 7007.392529] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7015.297279] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7017.229727] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7023.919422] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8cd588c0d000.ost_server_uuid 50 [ 7025.557188] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8cd588c0d000.ost_server_uuid in FULL state after 0 sec [ 7030.753297] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7030.767062] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7030.785274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7031.510534] Lustre: server umount lustre-OST0000 complete [ 7037.922975] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7037.932237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7037.942107] Lustre: Skipped 1 previous similar message [ 7037.957842] Lustre: Skipped 1 previous similar message [ 7042.351247] Lustre: server umount lustre-MDT0000 complete [ 7043.041829] LustreError: 227690:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7043.067926] LustreError: 227690:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7045.843491] LustreError: 227671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783141807 with bad export cookie 11767869863859227738 [ 7045.854362] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7045.857909] LustreError: 227671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7056.538488] Lustre: server umount lustre-MDT0001 complete [ 7066.094629] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 7068.577178] Key type lgssc unregistered [ 7068.848863] LNet: 232900:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7068.857279] LNetError: 232900:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7068.871819] LNet: Removed LNI 192.168.202.104@tcp [ 7069.690967] Key type .llcrypt unregistered [ 7069.696677] Key type ._llcrypt unregistered [ 7096.677871] Key type ._llcrypt registered [ 7096.679277] Key type .llcrypt registered [ 7096.783994] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 7098.079209] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7098.266215] alg: No test for adler32 (adler32-zlib) [ 7099.308942] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 7099.559735] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 7101.247942] Key type lgssc registered [ 7102.332718] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7111.074775] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7111.095399] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7112.571445] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7116.242416] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7122.933767] LustreError: 234608:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7122.959502] LustreError: 234608:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7123.706165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7123.960850] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7127.998219] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7130.478535] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7137.085865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7137.471175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7138.480291] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7142.376570] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7151.395232] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7151.573923] Lustre: lustre-OST0001: new disk, initializing [ 7151.578363] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7151.586263] Lustre: 236914:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7151.693419] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7157.813358] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7161.403880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7161.413364] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7161.483708] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7168.866747] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7174.282189] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7186.748387] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7186.763652] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7186.779599] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7187.426414] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7187.433116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7187.442176] Lustre: Skipped 1 previous similar message [ 7187.469683] Lustre: Skipped 3 previous similar messages [ 7192.544619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7192.553293] Lustre: Skipped 3 previous similar messages [ 7193.344910] Lustre: server umount lustre-MDT0000 complete [ 7197.357862] LustreError: 234589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783141958 with bad export cookie 12158473021706057159 [ 7197.370627] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7197.374527] LustreError: 234589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7197.406042] LustreError: 235345:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7197.666679] LustreError: 234602:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7197.668712] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7197.682340] LustreError: 234602:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7197.708651] Lustre: Skipped 2 previous similar messages [ 7197.892613] Lustre: server umount lustre-MDT0001 complete [ 7204.055281] Lustre: server umount lustre-OST0000 complete [ 7207.800155] Lustre: server umount lustre-OST0001 complete [ 7228.796770] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 7239.058796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7239.588673] LustreError: 240025:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7239.600066] LustreError: 240025:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7239.675457] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7244.420442] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7244.773812] LustreError: 240026:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7249.893656] LustreError: 240025:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7252.273703] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7252.638091] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7256.571619] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7259.545210] Lustre: 241166:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7266.695168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7273.018448] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7277.433153] LustreError: 241520:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7277.448571] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7281.059862] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7281.281705] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7281.290828] Lustre: Skipped 1 previous similar message [ 7285.401762] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7287.557226] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7295.508513] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7299.308863] Lustre: 243036:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7308.773116] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7308.784667] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7308.806993] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7314.430727] Lustre: server umount mylustre-MDT0000 complete [ 7316.458051] LustreError: 240025:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7316.479930] LustreError: 240025:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7317.996262] LustreError: 240000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783142079 with bad export cookie 12158473021706060022 [ 7317.997092] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7318.007328] LustreError: 240000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7318.450449] Lustre: server umount mylustre-MDT0001 complete [ 7332.151253] Lustre: server umount mylustre-OST0000 complete [ 7345.379535] Lustre: server umount mylustre-OST0001 complete [ 7364.736992] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 7374.802776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7375.407084] LustreError: 245344:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7375.517724] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7380.173433] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7388.373111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7392.703823] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7395.213897] Lustre: 246485:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7401.149572] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7406.897715] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7413.830062] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7413.867389] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7420.006482] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7420.423442] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7426.665534] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7429.753575] Lustre: 248354:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7435.748989] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7435.754918] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7435.768313] Lustre: Skipped 5 previous similar messages [ 7435.779249] Lustre: Skipped 5 previous similar messages [ 7441.792651] Lustre: server umount tfs-MDT0000 complete [ 7444.775357] LustreError: 245319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783142206 with bad export cookie 12158473021706061975 [ 7444.780452] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7444.783520] LustreError: 245319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7445.103047] Lustre: server umount tfs-MDT0001 complete [ 7459.055942] Lustre: server umount tfs-OST0000 complete [ 7471.938138] Lustre: server umount tfs-OST0001 complete [ 7491.349798] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 7500.576095] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7500.916345] LustreError: 250604:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7500.932287] LustreError: 250604:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7501.005803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7501.011609] Lustre: Skipped 3 previous similar messages [ 7504.771176] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7513.049879] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7516.979475] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7519.491748] Lustre: 251748:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7525.848677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7531.186528] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7537.463854] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7540.049884] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7546.390034] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7547.175582] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7553.510050] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7556.472869] Lustre: 253618:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7562.877413] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 01:18:43 (1783142323) [ 7580.129831] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7580.142105] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7580.154644] Lustre: Skipped 1 previous similar message [ 7580.165347] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7580.168214] Lustre: Skipped 6 previous similar messages [ 7584.529481] Lustre: server umount lustre-MDT0000 complete [ 7588.447451] LustreError: 250579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783142350 with bad export cookie 12158473021706063669 [ 7588.454414] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7588.463628] LustreError: 250579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7594.906739] Lustre: server umount lustre-MDT0001 complete [ 7604.661604] Lustre: server umount lustre-OST0000 complete [ 7614.329409] Lustre: server umount lustre-OST0001 complete [ 7621.169913] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 7631.231669] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 7666.264363] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7666.264363] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7666.264363] [ 7666.289313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7666.478884] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7666.502062] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7666.555789] Lustre: lustre-MDT0000: new disk, initializing [ 7666.617449] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7666.626355] Lustre: Skipped 3 previous similar messages [ 7666.644266] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7670.796866] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7681.002669] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7681.002669] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7681.002669] [ 7681.061271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7681.210031] Lustre: 256803:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7681.266498] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7681.270826] Lustre: Skipped 1 previous similar message [ 7681.328710] Lustre: lustre-MDT0001: new disk, initializing [ 7681.444773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7681.459744] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7685.375264] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7695.611852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7695.878348] Lustre: lustre-OST0000: new disk, initializing [ 7695.881151] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7695.893390] Lustre: 258386:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7697.201947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7697.223035] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7697.376066] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7702.833825] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7715.043837] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7715.256337] Lustre: lustre-OST0001: new disk, initializing [ 7715.260803] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7715.268900] Lustre: 259255:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7716.457723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7716.481754] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7716.637933] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7723.142340] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7735.654549] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 01:21:35 (1783142495) [ 7750.368738] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 01:21:50 (1783142510) [ 7768.043048] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7768.044710] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7768.049217] Lustre: Skipped 6 previous similar messages [ 7768.052017] Lustre: Skipped 8 previous similar messages [ 7770.427885] Lustre: server umount lustre-OST0000 complete [ 7773.174192] LustreError: 258370:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7773.190930] LustreError: 258370:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 7773.670110] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7779.979147] Lustre: server umount lustre-MDT0000 complete [ 7782.689512] LustreError: 256795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783142544 with bad export cookie 12158473021706065223 [ 7782.695375] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7782.697123] LustreError: 256795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7789.016952] Lustre: server umount lustre-MDT0001 complete [ 7797.232834] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 7806.680586] Lustre: server umount lustre-OST0001 complete [ 7812.264358] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 7824.272325] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 7852.378438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7852.584693] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7852.611849] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7852.711960] Lustre: lustre-MDT0000: new disk, initializing [ 7852.805242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7856.467601] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7865.774389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7865.844622] Lustre: 263788:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7866.046361] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7869.182255] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7876.261249] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7880.957922] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7887.177550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7887.492842] Lustre: lustre-OST0000: new disk, initializing [ 7887.498478] Lustre: Skipped 1 previous similar message [ 7887.502534] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7887.508510] Lustre: Skipped 2 previous similar messages [ 7887.514244] Lustre: 265757:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7889.537810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7889.548823] Lustre: Skipped 1 previous similar message [ 7889.555277] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7889.611519] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7892.738332] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7900.530806] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7910.368683] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7910.369869] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7910.374764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7910.383129] Lustre: Skipped 5 previous similar messages [ 7910.392801] Lustre: Skipped 8 previous similar messages [ 7912.640441] Lustre: server umount lustre-OST0000 complete [ 7922.936975] Lustre: server umount lustre-MDT0000 complete [ 7926.109532] LustreError: 263780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783142687 with bad export cookie 12158473021706066686 [ 7926.126485] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7926.339440] Lustre: server umount lustre-MDT0001 complete [ 7935.286690] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 7937.661815] Key type lgssc unregistered [ 7937.905568] LNet: 267516:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7937.918112] LNetError: 267516:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7938.994103] LNet: Removed LNI 192.168.202.104@tcp [ 7940.018497] Key type .llcrypt unregistered [ 7940.026429] Key type ._llcrypt unregistered [ 7965.319339] Key type ._llcrypt registered [ 7965.323593] Key type .llcrypt registered [ 7965.474712] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 7966.549820] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7966.676812] alg: No test for adler32 (adler32-zlib) [ 7967.712590] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 7967.927340] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 7969.599277] Key type lgssc registered [ 7970.656590] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7976.552716] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7976.567983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7977.879266] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7982.443347] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 7988.214673] LustreError: 268978:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7988.241200] LustreError: 268978:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7990.741758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7991.192645] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7994.799170] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8000.788218] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8005.557573] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8010.720439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8011.037313] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8015.937655] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8022.562678] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8023.351277] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8023.553910] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8026.592628] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8026.596292] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8026.622349] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8028.642720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8028.647494] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8028.661174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8031.464446] Lustre: server umount lustre-OST0000 complete [ 8037.345805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8037.353938] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8037.365464] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8037.368199] Lustre: Skipped 1 previous similar message [ 8041.351581] Lustre: server umount lustre-MDT0000 complete [ 8044.003194] LustreError: 268979:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8044.029638] LustreError: 268979:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8044.488678] LustreError: 270821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783142806 with bad export cookie 12062306474449433596 [ 8044.489326] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8044.496195] LustreError: 270821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8044.840338] Lustre: server umount lustre-MDT0001 complete [ 8053.497512] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 8055.670641] Key type lgssc unregistered [ 8055.907123] LNet: 272562:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8055.918896] LNetError: 272562:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8055.938870] LNet: Removed LNI 192.168.202.104@tcp [ 8056.636541] Key type .llcrypt unregistered [ 8056.641681] Key type ._llcrypt unregistered [ 8069.941707] Key type ._llcrypt registered [ 8069.943603] Key type .llcrypt registered [ 8070.034031] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8071.121326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8071.131907] alg: No test for adler32 (adler32-zlib) [ 8072.142201] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 8072.330314] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 8073.975920] Key type lgssc registered [ 8074.948686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8082.568920] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8082.585161] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8098.080459] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8103.199441] LustreError: 273736:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.104@tcp: failed processing log, type 4: rc = -110 [ 8129.887409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8135.369618] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8142.169545] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8144.321357] Lustre: server umount lustre-OST0000 complete [ 8158.651825] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 8161.377918] Key type lgssc unregistered [ 8161.674833] LNet: 275368:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8161.688834] LNetError: 275368:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8161.707354] LNet: Removed LNI 192.168.202.104@tcp [ 8162.433836] Key type .llcrypt unregistered [ 8162.438437] Key type ._llcrypt unregistered [ 8190.863706] Key type ._llcrypt registered [ 8190.865867] Key type .llcrypt registered [ 8190.948789] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8191.740178] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8191.779498] alg: No test for adler32 (adler32-zlib) [ 8192.760575] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 8192.958608] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 8194.607170] Key type lgssc registered [ 8195.286980] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8204.084715] loop0: detected capacity change from 0 to 204800000 [ 8204.189798] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8204.196074] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8204.384990] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8205.180778] loop0: detected capacity change from 0 to 204800000 [ 8205.344210] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8205.351302] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8205.668347] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8206.411481] loop0: detected capacity change from 0 to 204800000 [ 8206.577459] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8206.591681] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8206.680121] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8207.445397] loop0: detected capacity change from 0 to 204800000 [ 8207.592491] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8208.363755] loop0: detected capacity change from 0 to 204800000 [ 8208.422788] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8211.317698] loop0: detected capacity change from 0 to 204800000 [ 8211.349048] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8214.419913] loop0: detected capacity change from 0 to 204800000 [ 8214.439253] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8217.431994] loop0: detected capacity change from 0 to 204800000 [ 8217.456967] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8220.655291] loop0: detected capacity change from 0 to 204800000 [ 8220.751969] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8220.832840] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8220.886269] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8220.896907] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8220.907337] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8220.917845] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8220.938083] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8220.956140] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8220.974587] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8220.987532] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8220.999576] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8221.011150] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8221.021545] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8221.032591] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8221.040769] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8221.046970] Lustre: 278758:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8225.988429] Lustre: server umount MGS complete [ 8226.652769] loop0: detected capacity change from 0 to 204800000 [ 8226.769836] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8227.046199] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8227.114371] Lustre: lustre-MDT0000: Aborting client recovery [ 8227.826029] loop1: detected capacity change from 0 to 204800000 [ 8227.925730] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8228.006228] Lustre: 279189:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8228.013563] Lustre: 279189:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8228.021395] Lustre: 279189:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8228.033835] Lustre: 279189:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8228.046391] Lustre: 279189:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8228.062893] Lustre: 279189:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8228.083332] Lustre: 279189:0:(osd_scrub.c:1855:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8228.114985] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8228.277463] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8228.318785] Lustre: lustre-MDT0001: Aborting client recovery [ 8229.019857] loop2: detected capacity change from 0 to 204800000 [ 8229.174071] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8229.299520] Lustre: 279267:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8229.321275] Lustre: 279267:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8229.343940] Lustre: 279267:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8229.359271] Lustre: 279267:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8229.368590] Lustre: 279267:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8229.378470] Lustre: 279267:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8229.386879] Lustre: 279267:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8229.498643] Lustre: Found index 0 for lustre-OST0000, updating log [ 8229.535942] Lustre: 279286:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8229.600700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8229.615502] Lustre: lustre-OST0000: Aborting recovery [ 8230.248105] loop3: detected capacity change from 0 to 204800000 [ 8230.425255] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8230.472711] Lustre: 279351:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8230.478736] Lustre: 279351:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8230.487321] Lustre: 279351:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8230.493867] Lustre: 279351:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8230.500237] Lustre: 279351:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8230.506271] Lustre: 279351:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8230.513519] Lustre: 279351:0:(osd_scrub.c:1855:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8230.534102] Lustre: Found index 1 for lustre-OST0001, updating log [ 8230.553215] Lustre: 279356:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8230.618602] Lustre: lustre-OST0001: Aborting recovery [ 8236.025924] LustreError: 279133:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8237.026764] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8237.030362] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8237.051773] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8237.071908] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8238.069551] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8238.071270] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8238.693509] LustreError: 279100:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 240us: rc = -2 [ 8239.202033] LustreError: 279097:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 677us: rc = -2 [ 8239.217797] LustreError: 279097:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 38 previous similar messages [ 8253.408530] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8253.410191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8253.423544] Lustre: Skipped 1 previous similar message [ 8253.432442] Lustre: Skipped 3 previous similar messages [ 8257.310611] Lustre: server umount lustre-MDT0000 complete [ 8258.031727] LustreError: 279082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783143019 with bad export cookie 6309923940583721878 [ 8258.034949] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8258.046188] LustreError: 279082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8258.527976] LustreError: 279098:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8258.528794] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8258.531495] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8258.551308] LustreError: 279098:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8258.588633] Lustre: Skipped 3 previous similar messages [ 8263.648468] LustreError: 279097:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8263.654503] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8263.670389] LustreError: 279097:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8263.679520] Lustre: Skipped 2 previous similar messages [ 8267.743810] LustreError: 279587:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8267.746173] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8267.754319] LustreError: 279587:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8267.765143] Lustre: Skipped 1 previous similar message [ 8272.351448] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8272.603537] Lustre: server umount lustre-MDT0001 complete [ 8279.528324] Lustre: server umount lustre-OST0000 complete [ 8286.452028] Lustre: server umount lustre-OST0001 complete [ 8291.676976] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 01:30:52 (1783143052) [ 8310.698392] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 8316.760354] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8339.091807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8339.225250] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8339.245378] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8339.290589] Lustre: lustre-MDT0000: new disk, initializing [ 8339.337686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8339.347503] Lustre: Skipped 1 previous similar message [ 8339.362750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8342.006297] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8350.696422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8350.797365] Lustre: 283421:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8350.861322] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8350.863781] Lustre: Skipped 1 previous similar message [ 8350.917201] Lustre: lustre-MDT0001: new disk, initializing [ 8350.979343] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8351.027699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8351.037169] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8354.108230] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8361.063335] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8364.432154] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8368.812386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8368.993170] Lustre: lustre-OST0000: new disk, initializing [ 8368.995725] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8369.002278] Lustre: 285387:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8369.059614] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8373.198882] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8379.416322] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8379.429117] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8379.505460] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8379.514443] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8383.327224] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8388.988319] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8393.615093] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8401.725722] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8444.389106] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8444.412495] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8444.422764] Lustre: Skipped 1 previous similar message [ 8454.626065] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8454.629398] Lustre: Skipped 2 previous similar messages [ 8458.207131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8458.315210] Lustre: server umount lustre-OST0000 complete [ 8463.840253] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8463.843623] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8463.855553] Lustre: Skipped 1 previous similar message [ 8466.532402] Lustre: server umount lustre-MDT0000 complete [ 8469.146038] LustreError: 285397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783143230 with bad export cookie 6309923940583724566 [ 8469.149428] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8469.162572] LustreError: 285397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8469.408677] Lustre: server umount lustre-MDT0001 complete [ 8475.080855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8477.696882] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8480.619557] Lustre: server umount MGS complete [ 8487.738940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8487.992430] LustreError: 289154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8488.014473] LustreError: 289167:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8488.048964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8490.882922] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8493.538655] LustreError: 289155:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8493.554798] LustreError: 289155:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8496.306140] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8499.280100] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8503.405764] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8507.048838] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8511.298858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8515.509996] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8521.858813] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8542.179696] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8542.198945] Lustre: Skipped 1 previous similar message [ 8542.204180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8542.209954] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8542.210595] Lustre: Skipped 3 previous similar messages [ 8545.910995] Lustre: server umount lustre-OST0000 complete [ 8552.416852] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8552.424917] Lustre: Skipped 1 previous similar message [ 8554.343936] Lustre: server umount lustre-MDT0000 complete [ 8556.958055] LustreError: 289135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783143318 with bad export cookie 6309923940583726414 [ 8556.959238] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8556.966840] LustreError: 289135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8557.161095] Lustre: server umount lustre-MDT0001 complete [ 8563.831611] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 8565.628665] Key type lgssc unregistered [ 8565.827446] LNet: 293096:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8565.830790] LNetError: 293096:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8565.843821] LNet: Removed LNI 192.168.202.104@tcp [ 8566.342370] Key type .llcrypt unregistered [ 8566.343974] Key type ._llcrypt unregistered [ 8590.738672] Key type ._llcrypt registered [ 8590.742442] Key type .llcrypt registered [ 8590.794962] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 8601.053538] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8601.906531] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8601.923375] alg: No test for adler32 (adler32-zlib) [ 8602.867512] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 8603.007124] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 8604.647202] Key type lgssc registered [ 8605.399696] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8628.091537] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8628.118257] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8629.262101] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8629.276690] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8629.323657] Lustre: lustre-MDT0000: new disk, initializing [ 8629.383989] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8629.394619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8632.132316] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8639.023488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8639.139868] Lustre: 297029:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8639.186563] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8639.189148] Lustre: Skipped 1 previous similar message [ 8639.242743] Lustre: lustre-MDT0001: new disk, initializing [ 8639.303577] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8639.338816] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8639.343772] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8642.115229] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8648.582733] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8652.157583] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8656.607268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8656.847918] Lustre: lustre-OST0000: new disk, initializing [ 8656.851484] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8656.854786] Lustre: 298994:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8656.910440] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8661.015889] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8666.660258] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8666.666688] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8666.734955] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8666.992856] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8672.957871] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8675.264485] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8684.210963] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8693.343137] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8728.034494] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8728.036068] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8728.043614] Lustre: Skipped 1 previous similar message [ 8728.044091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8728.052743] Lustre: Skipped 1 previous similar message [ 8733.160990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8733.165838] Lustre: Skipped 1 previous similar message [ 8738.277914] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8738.281668] Lustre: Skipped 1 previous similar message [ 8742.367121] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8742.465996] Lustre: server umount lustre-OST0000 complete [ 8746.976513] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8746.980317] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8746.986781] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8748.525411] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8753.632311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8753.635511] Lustre: Skipped 3 previous similar messages [ 8759.263121] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8759.371812] Lustre: server umount lustre-MDT0000 complete [ 8761.372052] LustreError: 300499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783143523 with bad export cookie 9603828660189511391 [ 8761.379965] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8761.592285] Lustre: server umount lustre-MDT0001 complete [ 8766.382668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8768.690570] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8771.389885] Lustre: server umount MGS complete [ 8777.867749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8778.188140] LustreError: 302826:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8778.244646] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8780.580959] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8783.331938] LustreError: 302814:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8783.350573] LustreError: 302814:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8785.661179] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8785.852635] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8788.328336] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8792.308801] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8795.472597] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8799.418166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8799.628972] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8803.087689] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8806.924501] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8826.340723] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8826.342120] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8826.342153] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8826.342158] Lustre: Skipped 2 previous similar messages [ 8826.354508] Lustre: Skipped 1 previous similar message [ 8830.275101] Lustre: server umount lustre-OST0000 complete [ 8836.579027] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8838.404945] Lustre: server umount lustre-MDT0000 complete [ 8840.510162] LustreError: 302795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783143602 with bad export cookie 9603828660189513246 [ 8840.511723] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8840.523444] LustreError: 302795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8840.721730] Lustre: server umount lustre-MDT0001 complete [ 8846.606697] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 8848.171381] Key type lgssc unregistered [ 8848.366808] LNet: 306751:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8848.370864] LNetError: 306751:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8849.446463] LNet: Removed LNI 192.168.202.104@tcp [ 8849.953448] Key type .llcrypt unregistered [ 8849.954698] Key type ._llcrypt unregistered [ 8860.832907] Key type ._llcrypt registered [ 8860.834832] Key type .llcrypt registered [ 8860.900183] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 8868.802762] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8869.302455] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8869.396292] alg: No test for adler32 (adler32-zlib) [ 8870.301702] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 8870.439233] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 8872.047284] Key type lgssc registered [ 8872.664246] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8886.780653] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8895.976466] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8895.995242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8897.124445] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8897.145447] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8897.191912] Lustre: lustre-MDT0000: new disk, initializing [ 8897.254655] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8897.263148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8899.273381] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8905.214722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8905.348846] Lustre: lustre-OST0000: new disk, initializing [ 8905.352459] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8905.354132] Lustre: Skipped 1 previous similar message [ 8905.356734] Lustre: 310773:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8905.391662] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8908.499847] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8911.900469] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8911.910734] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8911.926460] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9280.480465] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9280.490041] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9283.425893] Lustre: server umount lustre-MDT0000 complete [ 9285.191046] LustreError: 309961:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144046 with bad export cookie 1161634819920329632 [ 9285.192739] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9285.198761] LustreError: 309961:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9285.271652] Lustre: server umount lustre-OST0000 complete [ 9297.035281] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 9301.419816] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9317.659490] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9317.746365] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9317.759219] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9317.797390] Lustre: lustre-MDT0000: new disk, initializing [ 9317.835911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9317.846146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9319.554559] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9325.331814] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9325.375401] Lustre: 315025:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9325.410719] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9325.415912] Lustre: Skipped 1 previous similar message [ 9325.462287] Lustre: lustre-MDT0001: new disk, initializing [ 9325.509991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9325.532364] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9325.536495] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9327.371899] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9331.723343] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9333.959539] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9337.046760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9337.189334] Lustre: lustre-OST0000: new disk, initializing [ 9337.191494] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9337.194745] Lustre: 316992:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9337.238261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9339.970710] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9344.029730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9344.038277] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9344.110517] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9344.761833] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9346.893081] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9347.004682] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9348.836686] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9348.952561] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9354.209787] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9354.210365] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9354.214222] LustreError: Skipped 1 previous similar message [ 9354.219416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9354.222735] Lustre: Skipped 1 previous similar message [ 9356.114062] Lustre: server umount lustre-OST0000 complete [ 9359.328769] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9359.331716] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9359.335145] Lustre: Skipped 2 previous similar messages [ 9363.736870] Lustre: server umount lustre-MDT0000 complete [ 9365.457988] LustreError: 315020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144127 with bad export cookie 1161634819923270423 [ 9365.459586] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9365.463282] LustreError: 315020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9365.616320] Lustre: server umount lustre-MDT0001 complete [ 9367.380981] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9368.114233] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 01:48:48 (1783144128) [ 9371.303713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9371.504092] LustreError: 319301:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9371.513851] LustreError: 319301:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9371.535481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9373.231683] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9376.737307] LustreError: 319302:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9377.408541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9379.315618] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9382.632852] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9384.861477] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9387.977593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9388.104487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9388.108027] Lustre: Skipped 1 previous similar message [ 9390.797609] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9394.239932] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9397.703187] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9397.780634] Lustre: lustre-OST0001: new disk, initializing [ 9397.784099] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9397.788601] Lustre: 322135:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9400.317170] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9404.985407] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9405.107128] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9406.494291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9406.499548] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9406.524915] LustreError: 323080:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9407.589268] LustreError: 323086:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9407.599678] LustreError: 323086:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9407.893472] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8cd585554800.ost_server_uuid 50 [ 9408.537273] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8cd585554800.ost_server_uuid in FULL state after 0 sec [ 9408.610418] LustreError: 323080:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9408.614290] LustreError: 323080:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9410.658406] LustreError: 323080:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9410.663375] LustreError: 323080:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9410.774282] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8cd585554800.ost_server_uuid 50 [ 9411.439961] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8cd585554800.ost_server_uuid in FULL state after 0 sec [ 9412.712671] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9424.037607] Lustre: lustre-OST0001: Not available for connect from 192.168.202.4@tcp (stopping) [ 9424.040378] Lustre: Skipped 1 previous similar message [ 9426.916179] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9426.920817] Lustre: Skipped 1 previous similar message [ 9429.145760] Lustre: lustre-OST0001: Not available for connect from 192.168.202.4@tcp (stopping) [ 9429.148712] Lustre: Skipped 2 previous similar messages [ 9437.154821] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9437.157380] Lustre: Skipped 4 previous similar messages [ 9438.175095] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9438.224434] Lustre: server umount lustre-OST0001 complete [ 9442.273443] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9442.274632] LustreError: 321124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9442.279402] Lustre: Skipped 1 previous similar message [ 9442.287283] LustreError: 321124:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9446.369429] Lustre: server umount lustre-OST0000 complete [ 9449.439825] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9452.512736] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9452.518563] Lustre: Skipped 1 previous similar message [ 9457.632164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9457.634656] Lustre: Skipped 6 previous similar messages [ 9462.239109] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9462.320066] Lustre: server umount lustre-MDT0000 complete [ 9462.757275] LustreError: 319298:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9462.763625] LustreError: 319298:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9464.313637] LustreError: 319282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144225 with bad export cookie 1161634819923271291 [ 9464.315443] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9464.318895] LustreError: 319282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9464.464931] Lustre: server umount lustre-MDT0001 complete [ 9469.715169] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 9470.997434] Key type lgssc unregistered [ 9471.157560] LNet: 324822:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9471.162841] LNetError: 324822:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9471.177200] LNet: Removed LNI 192.168.202.104@tcp [ 9471.555194] Key type .llcrypt unregistered [ 9471.556548] Key type ._llcrypt unregistered [ 9481.856429] Key type ._llcrypt registered [ 9481.857721] Key type .llcrypt registered [ 9481.899463] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9482.248814] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9482.292713] alg: No test for adler32 (adler32-zlib) [ 9483.152146] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 9483.245551] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 9484.831189] Key type lgssc registered [ 9485.240317] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9488.037416] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9488.045677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9489.194172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9490.576249] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9493.107182] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9495.714482] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9495.891275] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9497.613406] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9501.319658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9501.466175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9503.698190] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9505.572755] LustreError: 327897:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9505.582651] LustreError: 327897:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9506.482286] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9509.033523] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9509.146840] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9511.237649] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9513.960372] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9514.289026] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9514.290698] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9520.738333] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9520.818622] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9601.505695] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9601.506802] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9601.508810] LustreError: Skipped 1 previous similar message [ 9601.514183] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9601.517111] Lustre: Skipped 1 previous similar message [ 9604.666088] Lustre: server umount lustre-OST0001 complete [ 9606.624439] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9606.625184] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9606.628845] LustreError: 328489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9606.633422] Lustre: Skipped 2 previous similar messages [ 9606.633917] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9606.640296] LustreError: 328489:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9606.641697] Lustre: Skipped 1 previous similar message [ 9611.744693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9611.746353] LustreError: 328489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9611.746953] Lustre: Skipped 1 previous similar message [ 9611.752361] LustreError: 328489:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9612.366188] Lustre: server umount lustre-OST0000 complete [ 9613.793122] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9613.795525] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9613.800085] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9616.865579] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9619.672764] Lustre: server umount lustre-MDT0000 complete [ 9620.931177] LustreError: 329902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144382 with bad export cookie 17443575367644810975 [ 9620.933964] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9620.936082] LustreError: 329902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9621.060953] Lustre: server umount lustre-MDT0001 complete [ 9624.770823] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 9625.786984] Key type lgssc unregistered [ 9625.928814] LNet: 331155:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9625.935665] LNetError: 331155:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9625.948348] LNet: Removed LNI 192.168.202.104@tcp [ 9626.289370] Key type .llcrypt unregistered [ 9626.290592] Key type ._llcrypt unregistered [ 9647.035026] Key type ._llcrypt registered [ 9647.036693] Key type .llcrypt registered [ 9647.078324] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9647.580017] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9647.589939] alg: No test for adler32 (adler32-zlib) [ 9648.453870] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 9648.562062] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 9650.151188] Key type lgssc registered [ 9650.588969] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9654.497520] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9654.516628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9655.712459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9657.209470] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9660.561638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9660.720427] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9662.461164] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9663.616868] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9666.402976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9666.571879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9668.775359] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9670.691353] LustreError: 334905:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9670.698151] LustreError: 334905:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9672.314909] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9672.404364] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9674.661516] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9680.875929] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9680.876887] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9683.434062] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9687.169388] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9691.108544] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9691.109858] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9691.113613] Lustre: Skipped 2 previous similar messages [ 9691.116229] Lustre: Skipped 1 previous similar message [ 9696.227262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9696.235703] Lustre: Skipped 3 previous similar messages [ 9697.005052] Lustre: server umount lustre-MDT0000 complete [ 9698.412244] LustreError: 333400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144460 with bad export cookie 5449045873340497457 [ 9698.415842] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9698.417257] LustreError: 333400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9698.555861] Lustre: server umount lustre-MDT0001 complete [ 9709.514723] Lustre: server umount lustre-OST0000 complete [ 9721.255750] Lustre: server umount lustre-OST0001 complete [ 9727.589024] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9731.090058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9731.242081] LustreError: 338680:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9731.271863] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9732.790079] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9735.969013] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9737.580823] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9738.603444] Lustre: 339821:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9740.920151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9741.056617] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9741.060180] Lustre: Skipped 1 previous similar message [ 9742.115466] LustreError: 340173:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9742.118474] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9743.081536] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9746.033762] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9747.187342] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9748.201290] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9751.570957] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9752.856790] Lustre: 341687:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9756.640981] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9756.644176] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9756.651066] Lustre: Skipped 1 previous similar message [ 9756.655496] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9757.667169] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9757.671085] Lustre: Skipped 1 previous similar message [ 9760.029143] Lustre: server umount lustre-MDT0000 complete [ 9761.489732] LustreError: 339423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144523 with bad export cookie 5449045873340499144 [ 9761.494340] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9761.496392] LustreError: 339423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9761.613951] Lustre: server umount lustre-MDT0001 complete [ 9773.314267] Lustre: server umount lustre-OST0000 complete [ 9785.018392] Lustre: server umount lustre-OST0001 complete [ 9786.497307] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9787.057950] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9787.677286] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 01:55:48 (1783144548) [ 9796.054979] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9799.305765] loop0: detected capacity change from 0 to 1099511627776 [ 9799.461444] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9799.464590] *** VALIDATE xfs *** [ 9799.469346] XFS (loop0): Mounting V5 Filesystem [ 9799.478688] XFS (loop0): Ending clean mount [ 9799.929901] loop1: detected capacity change from 0 to 18691697672192 [ 9821.412468] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 9824.947363] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9836.646209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9836.731292] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9836.742792] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9836.777081] Lustre: lustre-MDT0000: new disk, initializing [ 9836.796859] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9836.799412] Lustre: Skipped 1 previous similar message [ 9836.804753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9838.088163] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9842.228031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9842.277178] Lustre: 346795:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9842.301962] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9842.304030] Lustre: Skipped 1 previous similar message [ 9842.339304] Lustre: lustre-MDT0001: new disk, initializing [ 9842.383203] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9842.386897] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9843.709656] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9847.362653] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9849.136777] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9851.445792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9851.542907] Lustre: lustre-OST0000: new disk, initializing [ 9851.545077] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9851.548668] Lustre: 348765:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9853.112927] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9853.116835] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9853.146575] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9853.455331] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9857.204181] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9858.618446] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9858.694564] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9860.027102] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9860.098514] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9862.807138] Lustre: server umount lustre-OST0000 complete [ 9868.255878] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9868.259095] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9868.263411] Lustre: Skipped 1 previous similar message [ 9868.265908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9868.268108] Lustre: Skipped 4 previous similar messages [ 9873.888923] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9873.891934] Lustre: Skipped 2 previous similar messages [ 9873.925899] Lustre: server umount lustre-MDT0000 complete [ 9875.187274] LustreError: 346787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144636 with bad export cookie 5449045873340500677 [ 9875.190524] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9875.191582] LustreError: 346787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9884.689498] Lustre: server umount lustre-MDT0001 complete [ 9887.405617] XFS (loop0): Unmounting Filesystem [ 9889.168892] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 01:57:30 (1783144650) [ 9891.512520] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9891.651097] LustreError: 351318:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9891.676183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9891.679350] Lustre: Skipped 2 previous similar messages [ 9892.993217] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9895.842799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9897.365560] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9899.816810] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9901.560893] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9903.812225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9905.829230] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9908.316563] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9922.530293] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9922.531236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9922.532471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9922.536626] Lustre: Skipped 2 previous similar messages [ 9923.596435] Lustre: server umount lustre-OST0000 complete [ 9926.623896] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9930.899334] Lustre: server umount lustre-MDT0000 complete [ 9932.114297] LustreError: 351300:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783144693 with bad export cookie 5449045873340501559 [ 9932.119101] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9932.122223] LustreError: 351300:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9932.231388] Lustre: server umount lustre-MDT0001 complete [ 9935.643883] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 9936.602435] Key type lgssc unregistered [ 9936.720350] LNet: 354964:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9936.723270] LNetError: 354964:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9936.729291] LNet: Removed LNI 192.168.202.104@tcp [ 9937.037131] Key type .llcrypt unregistered [ 9937.038852] Key type ._llcrypt unregistered [ 9946.885403] Key type ._llcrypt registered [ 9946.886488] Key type .llcrypt registered [ 9946.922515] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9947.256430] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9947.307846] alg: No test for adler32 (adler32-zlib) [ 9948.159961] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 9948.249418] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 9949.831136] Key type lgssc registered [ 9950.192842] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9952.757329] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9952.764447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9953.902037] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9955.276057] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9958.516716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9958.703033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9960.186369] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9962.940165] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9964.967157] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9967.349168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9967.484680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9969.610321] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9972.728920] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9988.256943] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [ 9989.090681] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9989.097569] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9989.099862] Lustre: Skipped 1 previous similar message [ 9993.367108] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [ 9998.486116] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [ 9998.488810] Lustre: Skipped 2 previous similar messages [ 9999.839096] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9999.898955] Lustre: server umount lustre-MDT0001 complete [10003.607265] LustreError: 357240:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.4@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10003.613839] LustreError: 357240:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10003.737929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10003.781629] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10003.783730] Lustre: Found index 1 for lustre-MDT0001, updating log [10003.799062] Lustre: 356496:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10003.914376] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10005.566635] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10009.060700] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10013.668906] LustreError: 360340:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10013.671844] LustreError: 360340:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10013.675683] LustreError: 356499:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10013.693788] LustreError: 360343:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.104@tcp: cfg command failed: rc = -17 [10013.697775] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10013.697775] [10310.819847] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [10310.823203] Lustre: Skipped 2 previous similar messages [10311.137496] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10311.137683] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10311.141925] Lustre: Skipped 1 previous similar message [10313.294432] Lustre: server umount lustre-MDT0001 complete [10315.927656] LustreError: 356501:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.4@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10316.684254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10316.716815] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10316.718342] Lustre: Found index 1 for lustre-MDT0001, updating log [10316.729297] Lustre: 356496:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10316.837351] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10318.192850] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10321.892668] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10326.501942] LustreError: 372874:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10326.505121] LustreError: 372874:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10326.509377] LustreError: 356499:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10326.511714] LustreError: 356499:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10326.528177] LustreError: 372877:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [10326.531804] LustreError: 372877:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [10326.534864] LustreError: 372877:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.104@tcp: cfg command failed: rc = -17 [10326.537422] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.104@tcp [10326.537422] [10623.133063] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [10623.135637] Lustre: Skipped 2 previous similar messages [10623.972048] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10623.972378] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10623.978898] Lustre: Skipped 1 previous similar message [10626.837755] Lustre: server umount lustre-MDT0001 complete [10628.245905] LustreError: 356502:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.4@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10628.250879] LustreError: 356502:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10630.406591] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10630.436379] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10630.438181] Lustre: Found index 1 for lustre-MDT0001, updating log [10630.449734] Lustre: 356496:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10630.548135] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10631.778684] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10635.750135] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10638.308475] LustreError: 385595:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10638.311028] LustreError: 385595:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10638.313985] LustreError: 356499:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10638.317115] LustreError: 356499:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10638.333567] LustreError: 385598:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10638.338621] LustreError: 385598:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.104@tcp: cfg command failed: rc = -114 [10638.341951] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10638.341951] [10948.066181] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10948.066963] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10948.067259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10948.067264] Lustre: Skipped 2 previous similar messages [10948.070968] Lustre: Skipped 2 previous similar messages [10951.505288] Lustre: server umount lustre-MDT0000 complete [10952.687204] LustreError: 356487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783145714 with bad export cookie 17538856323037368801 [10952.690492] LustreError: 356487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10952.690770] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10953.183855] LustreError: 356501:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10953.184170] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10953.188932] LustreError: 356501:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10953.193339] Lustre: Skipped 1 previous similar message [10957.217086] LustreError: 358994:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10958.845435] Lustre: server umount lustre-MDT0001 complete [10960.215919] Lustre: server umount lustre-OST0000 complete [10963.177560] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [10966.119876] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [10977.245660] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 02:15:38 (1783145738) [10979.415284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10979.500259] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10979.509687] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10979.548886] Lustre: lustre-MDT0000: new disk, initializing [10979.572046] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10979.578402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10980.949169] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10985.174788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10985.222778] Lustre: 400154:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10985.248713] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10985.251825] Lustre: Skipped 1 previous similar message [10985.294573] Lustre: lustre-MDT0001: new disk, initializing [10985.330866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10985.346204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10985.350709] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10986.750237] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10990.473732] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10992.293668] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10994.454982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10994.561624] Lustre: lustre-OST0000: new disk, initializing [10994.563373] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10994.565829] Lustre: 402124:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10994.595956] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10996.450904] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [11000.260693] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11003.382060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11003.385405] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11003.415369] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11008.481949] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11008.483259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11008.485729] Lustre: Skipped 2 previous similar messages [11008.487721] Lustre: Skipped 5 previous similar messages [11014.177943] Lustre: server umount lustre-MDT0000 complete [11015.391350] LustreError: 400952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783145777 with bad export cookie 17538856323037370740 [11015.394154] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11015.394399] LustreError: 400964:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [11015.394526] LustreError: 400964:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000000a89848 x1869762746744448/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [11015.396672] LustreError: 400952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11015.525321] Lustre: server umount lustre-MDT0001 complete [11027.097789] Lustre: server umount lustre-OST0000 complete [11036.717474] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [11040.291282] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [11050.803714] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10859 sec ========= 02:16:51 (1783145811) [11051.405175] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 02:16:52 (1783145812) === [11052.486420] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 02:16:53 (1783145813) === [11064.828584] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [11065.769335] Key type lgssc unregistered [11065.887313] LNet: 408476:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11065.890170] LNetError: 408476:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11065.901288] LNet: Removed LNI 192.168.202.104@tcp [11066.177961] Key type .llcrypt unregistered [11066.179119] Key type ._llcrypt unregistered