[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 508190253 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003179] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008024] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010139] LSM: Security Framework initializing [ 0.011057] Yama: becoming mindful. [ 0.012045] SELinux: Initializing. [ 0.013080] *** VALIDATE selinux *** [ 0.021771] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025871] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027043] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028120] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029129] *** VALIDATE tmpfs *** [ 0.031414] *** VALIDATE proc *** [ 0.032253] *** VALIDATE cgroup *** [ 0.033012] *** VALIDATE cgroup2 *** [ 0.034285] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035149] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038025] Spectre V2 : User space: Vulnerable [ 0.039011] Speculative Store Bypass: Vulnerable [ 0.041979] debug: unmapping init [mem 0xffffffff8ea59000-0xffffffff8ea60fff] [ 0.043226] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044689] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045024] ... version: 2 [ 0.046017] ... bit width: 48 [ 0.047013] ... generic registers: 4 [ 0.048015] ... value mask: 0000ffffffffffff [ 0.049016] ... max period: 00007fffffffffff [ 0.050016] ... fixed-purpose events: 3 [ 0.051014] ... event mask: 000000070000000f [ 0.052330] rcu: Hierarchical SRCU implementation. [ 0.054532] smp: Bringing up secondary CPUs ... [ 0.055702] x86: Booting SMP configuration: [ 0.056027] .... node #0, CPUs: #1 #2 #3 [ 0.059763] smp: Brought up 1 node, 4 CPUs [ 0.061016] smpboot: Max logical packages: 1 [ 0.062022] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.128557] node 0 deferred pages initialised in 64ms [ 0.131012] devtmpfs: initialized [ 0.132280] x86/mm: Memory block size: 128MB [ 0.135448] gcov: version magic: 0x41383552 [ 0.138352] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.139077] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.140299] pinctrl core: initialized pinctrl subsystem [ 0.141214] [ 0.141849] ************************************************************* [ 0.142015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.143014] ** ** [ 0.144016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.145015] ** ** [ 0.146015] ** This means that this kernel is built to expose internal ** [ 0.147017] ** IOMMU data structures, which may compromise security on ** [ 0.148015] ** your system. ** [ 0.149015] ** ** [ 0.150016] ** If you see this message and you are not debugging the ** [ 0.151014] ** kernel, report this immediately to your vendor! ** [ 0.152017] ** ** [ 0.153011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.154013] ************************************************************* [ 0.155655] NET: Registered protocol family 16 [ 0.156364] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.157061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.158062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.159403] cpuidle: using governor menu [ 0.160000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.163529] PCI: Using configuration type 1 for base access [ 0.165130] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.173115] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.174014] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.176066] cryptd: max_cpu_qlen set to 1000 [ 0.177229] ACPI: Added _OSI(Module Device) [ 0.178012] ACPI: Added _OSI(Processor Device) [ 0.179013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.180016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.184482] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.188178] ACPI: Interpreter enabled [ 0.189077] ACPI: PM: (supports S0 S3 S4 S5) [ 0.190013] ACPI: Using IOAPIC for interrupt routing [ 0.191125] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.192403] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.203204] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.204038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.205021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.206188] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.208010] acpiphp: Slot [2] registered [ 0.209016] acpiphp: Slot [5] registered [ 0.210044] acpiphp: Slot [6] registered [ 0.211108] acpiphp: Slot [7] registered [ 0.212231] acpiphp: Slot [8] registered [ 0.213100] acpiphp: Slot [9] registered [ 0.214103] acpiphp: Slot [10] registered [ 0.215088] acpiphp: Slot [3] registered [ 0.215990] acpiphp: Slot [4] registered [ 0.216114] acpiphp: Slot [11] registered [ 0.217104] acpiphp: Slot [12] registered [ 0.218133] acpiphp: Slot [13] registered [ 0.219143] acpiphp: Slot [14] registered [ 0.220076] acpiphp: Slot [15] registered [ 0.221071] acpiphp: Slot [16] registered [ 0.222064] acpiphp: Slot [17] registered [ 0.223048] acpiphp: Slot [18] registered [ 0.224094] acpiphp: Slot [19] registered [ 0.225071] acpiphp: Slot [20] registered [ 0.226091] acpiphp: Slot [21] registered [ 0.227059] acpiphp: Slot [22] registered [ 0.228062] acpiphp: Slot [23] registered [ 0.229076] acpiphp: Slot [24] registered [ 0.230087] acpiphp: Slot [25] registered [ 0.231073] acpiphp: Slot [26] registered [ 0.232177] acpiphp: Slot [27] registered [ 0.233159] acpiphp: Slot [28] registered [ 0.234084] acpiphp: Slot [29] registered [ 0.235071] acpiphp: Slot [30] registered [ 0.236091] acpiphp: Slot [31] registered [ 0.237055] PCI host bridge to bus 0000:00 [ 0.237993] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.238019] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.239026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.240042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.242030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.244049] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.245245] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.248191] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.251000] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.260016] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.265062] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.269028] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.271030] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.274018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.277400] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.279605] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.280043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.282721] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.287015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.296017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.301015] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.304992] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.310016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.316016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.329019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.336601] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.342018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.348023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.363022] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.370559] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.378020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.383016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.398021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.409219] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.414017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.419020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.433017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.442724] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.449014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.455014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.470025] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.479214] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.483014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.489017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.500018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.511536] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.513402] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.516286] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.517360] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.519189] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.522106] iommu: Default domain type: Passthrough [ 0.524512] SCSI subsystem initialized [ 0.526194] ACPI: bus type USB registered [ 0.527095] usbcore: registered new interface driver usbfs [ 0.528076] usbcore: registered new interface driver hub [ 0.530078] usbcore: registered new device driver usb [ 0.531108] pps_core: LinuxPPS API ver. 1 registered [ 0.533013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.535065] PTP clock support registered [ 0.536277] EDAC MC: Ver: 3.0.0 [ 0.538138] PCI: Using ACPI for IRQ routing [ 0.539407] NetLabel: Initializing [ 0.540013] NetLabel: domain hash size = 128 [ 0.540932] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.543059] NetLabel: unlabeled traffic allowed by default [ 0.544252] vgaarb: loaded [ 0.546231] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.547012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.554337] clocksource: Switched to clocksource kvm-clock [ 0.652155] VFS: Disk quotas dquot_6.6.0 [ 0.653497] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.656497] *** VALIDATE ramfs *** [ 0.657883] *** VALIDATE hugetlbfs *** [ 0.659621] pnp: PnP ACPI init [ 0.662126] pnp: PnP ACPI: found 6 devices [ 0.678203] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.680541] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.682237] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.683760] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.686395] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.689331] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.692433] NET: Registered protocol family 2 [ 0.694794] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.699456] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.702232] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.707331] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.711232] TCP: Hash tables configured (established 65536 bind 65536) [ 0.714368] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.716874] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.718923] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.721404] NET: Registered protocol family 1 [ 0.724521] RPC: Registered named UNIX socket transport module. [ 0.726544] RPC: Registered udp transport module. [ 0.728115] RPC: Registered tcp transport module. [ 0.729781] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.732106] NET: Registered protocol family 44 [ 0.733516] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.734859] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.736455] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.738150] PCI: CLS 0 bytes, default 64 [ 0.739590] Unpacking initramfs... [ 2.147252] debug: unmapping init [mem 0xffff9b84fcc54000-0xffff9b84fffbffff] [ 2.151176] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.152954] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.155939] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.656481] Initialise system trusted keyrings [ 2.658570] Key type blacklist registered [ 2.660754] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.670244] zbud: loaded [ 2.673240] *** VALIDATE nfs *** [ 2.674354] *** VALIDATE nfs4 *** [ 2.675359] pstore: using deflate compression [ 2.678524] Platform Keyring initialized [ 2.781601] NET: Registered protocol family 38 [ 2.782874] Key type asymmetric registered [ 2.784299] Asymmetric key parser 'x509' registered [ 2.786365] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.789530] io scheduler mq-deadline registered [ 2.792168] io scheduler kyber registered [ 2.794521] io scheduler bfq registered [ 2.797582] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.801244] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.804904] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.807521] ACPI: Power Button [PWRF] [ 2.813128] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.820627] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.835416] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.842072] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.860860] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.889345] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.918794] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.923979] Non-volatile memory driver v1.3 [ 2.925379] Linux agpgart interface v0.103 [ 2.969364] virtio_blk virtio1: [vda] 134008 512-byte logical blocks (68.6 MB/65.4 MiB) [ 2.972739] vda: detected capacity change from 0 to 68612096 [ 2.999541] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.001530] vdb: detected capacity change from 0 to 1073741824 [ 3.015487] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.018521] vdc: detected capacity change from 0 to 2621440000 [ 3.030878] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.033588] vdd: detected capacity change from 0 to 2621440000 [ 3.048232] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.050530] vde: detected capacity change from 0 to 4294967296 [ 3.065041] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.067321] vdf: detected capacity change from 0 to 4294967296 [ 3.073505] libphy: Fixed MDIO Bus: probed [ 3.085388] usbcore: registered new interface driver usbserial_generic [ 3.087360] usbserial: USB Serial support registered for generic [ 3.089015] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.092611] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.093899] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.096062] mousedev: PS/2 mouse device common for all mice [ 3.098234] rtc_cmos 00:05: RTC can wake from S4 [ 3.100539] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.101645] rtc_cmos 00:05: registered as rtc0 [ 3.106289] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.108724] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.110376] intel_pstate: CPU model not supported [ 3.116491] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.117577] hid: raw HID events driver (C) Jiri Kosina [ 3.125246] usbcore: registered new interface driver usbhid [ 3.127393] usbhid: USB HID core driver [ 3.129087] drop_monitor: Initializing network drop monitor service [ 3.131660] Initializing XFRM netlink socket [ 3.133768] NET: Registered protocol family 10 [ 3.136912] Segment Routing with IPv6 [ 3.138787] NET: Registered protocol family 17 [ 3.141549] mpls_gso: MPLS GSO support [ 3.146623] RAS: Correctable Errors collector initialized. [ 3.149257] AVX version of gcm_enc/dec engaged. [ 3.151402] AES CTR mode by8 optimization enabled [ 3.249734] sched_clock: Marking stable (3249685354, 0)->(4219916347, -970230993) [ 3.254644] registered taskstats version 1 [ 3.257735] Loading compiled-in X.509 certificates [ 3.259695] zswap: loaded using pool lzo/zbud [ 3.285507] Key type big_key registered [ 3.300501] Key type encrypted registered [ 3.302791] ima: No TPM chip found, activating TPM-bypass! [ 3.304867] ima: Allocated hash algorithm: sha1 [ 3.306648] ima: No architecture policies found [ 3.308317] evm: Initialising EVM extended attributes: [ 3.310304] evm: security.selinux [ 3.311581] evm: security.ima [ 3.312685] evm: security.capability [ 3.314129] evm: HMAC attrs: 0x1 [ 3.316393] rtc_cmos 00:05: setting system clock to 2026-04-24 01:30:12 UTC (1776994212) [ 3.322675] debug: unmapping init [mem 0xffffffff8fa03000-0xffffffff8fbfffff] [ 3.325953] debug: unmapping init [mem 0xffffffff8e782000-0xffffffff8ea58fff] [ 3.338118] Write protecting the kernel read-only data: 28672k [ 3.341813] debug: unmapping init [mem 0xffffffff8ce03000-0xffffffff8cffffff] [ 3.344663] debug: unmapping init [mem 0xffffffff8d714000-0xffffffff8d7fffff] [ 3.380647] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.390153] systemd[1]: Detected virtualization kvm. [ 3.391738] systemd[1]: Detected architecture x86-64. [ 3.393253] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.421260] systemd[1]: No hostname configured. [ 3.422968] systemd[1]: Set hostname to . [ 3.424531] random: systemd: uninitialized urandom read (16 bytes read) [ 3.426392] systemd[1]: Initializing machine ID from random generator. [ 3.500421] random: ln: uninitialized urandom read (6 bytes read) [ 3.586315] random: systemd: uninitialized urandom read (16 bytes read) [ 3.588902] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.593204] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.596606] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.141741] device-mapper: uevent: version 1.0.3 [ 4.143567] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.782784] virtio_net virtio0 ens2: renamed from eth0 [ 4.810494] random: fast init done [ 4.838709] scsi host0: ata_piix [ 4.841957] scsi host1: ata_piix [ 4.844242] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.847218] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.838348] random: crng init done [ 9.844415] random: 7 urandom warning(s) missed due to ratelimiting [ 10.011065] dracut-initqueue[588]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 11.978165] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ 13.415061] hrtimer: interrupt took 3053296 ns [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.399630] printk: systemd: 26 output lines suppressed due to ratelimiting [ 17.739739] SELinux: Disabled at runtime. [ 17.862733] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 17.877032] systemd[1]: Detected virtualization kvm. [ 17.878626] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 19.519171] systemd[1]: initrd-switch-root.service: Succeeded. [ 19.530403] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 19.543047] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 19.558727] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 19.569318] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 19.590568] systemd[1]: Starting Journal Service... Starting Journal Service... [ 19.598144] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ 19.669567] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Kernel Debug File System... Mounting Huge Pages File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Initrd Root File System. Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 20.722851] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.471322] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.521191] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.994466] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 22.100163] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (7s / no limit)[ 27.175432] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (8s / no limit)[ 27.930878] NFS: Registering the id_resolver key type [ 27.934292] Key type id_resolver registered [ 27.938863] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (8s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Login Service. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg243-server login: [ 108.712830] libcfs: loading out-of-tree module taints kernel. [ 108.736164] Key type ._llcrypt registered [ 108.738157] Key type .llcrypt registered [ 108.829508] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 132.816642] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 134.727818] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 134.735494] alg: No test for adler32 (adler32-zlib) [ 136.168388] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 137.601405] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 139.416202] Key type lgssc registered [ 141.711774] Lustre: Echo OBD driver; http://www.lustre.org/ [ 164.526690] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 219.053724] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 235.426573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 235.483091] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 236.829895] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 236.858448] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 236.936576] Lustre: lustre-MDT0000: new disk, initializing [ 237.158623] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 237.195635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 242.428685] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 260.922329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 261.104744] Lustre: 6519:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 261.217805] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 261.224544] Lustre: Skipped 1 previous similar message [ 261.390760] Lustre: lustre-MDT0001: new disk, initializing [ 261.529864] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 261.582742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 261.607046] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 267.760455] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 274.068537] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 286.791597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 287.097714] Lustre: lustre-OST0000: new disk, initializing [ 287.104030] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 287.182694] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 293.008989] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 293.025974] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 293.132510] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 295.151307] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 311.998648] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 312.133638] Lustre: lustre-OST0001: new disk, initializing [ 312.139970] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 312.221467] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 320.553124] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 321.681611] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 321.698451] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 321.809216] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 333.038831] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 339.056239] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 350.826442] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing check_logdir /tmp/testlogs/ [ 358.363573] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing yml_node [ 363.808337] Lustre: DEBUG MARKER: Client: 2.17.52.54 [ 366.788071] Lustre: DEBUG MARKER: MDS: 2.17.52.54 [ 370.031808] Lustre: DEBUG MARKER: OSS: 2.17.52.54 [ 372.266777] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Apr 23 21:36:18 EDT 2026 [ 396.388250] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 398.487695] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 419.300598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 419.301604] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 419.320508] Lustre: Skipped 3 previous similar messages [ 419.330219] Lustre: Skipped 2 previous similar messages [ 422.523871] Lustre: server umount lustre-MDT0000 complete [ 426.132785] LustreError: 6513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776994635 with bad export cookie 17375300153031224144 [ 426.136160] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 426.146423] LustreError: 6513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 429.537767] LustreError: 6527:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 429.538095] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 429.538477] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 429.538483] Lustre: Skipped 1 previous similar message [ 429.548648] LustreError: 6527:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 6 previous similar messages [ 429.599514] Lustre: Skipped 1 previous similar message [ 431.588776] LustreError: 6528:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 431.592399] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 431.623923] Lustre: Skipped 2 previous similar messages [ 432.635907] Lustre: server umount lustre-MDT0001 complete [ 437.152395] Lustre: server umount lustre-OST0000 complete [ 441.442381] Lustre: server umount lustre-OST0001 complete [ 449.772513] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 459.513044] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 503.907954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 504.255208] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 504.298222] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 504.376448] Lustre: lustre-MDT0000: new disk, initializing [ 504.471154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 504.487940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 509.502357] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 521.417380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 521.684400] Lustre: 15001:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 521.797492] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 521.805823] Lustre: Skipped 1 previous similar message [ 521.952223] Lustre: lustre-MDT0001: new disk, initializing [ 522.117248] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 522.181676] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 522.205289] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 527.595416] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 537.772334] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 543.257971] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 551.377905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 552.005918] Lustre: lustre-OST0000: new disk, initializing [ 552.008628] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 552.182568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 553.331459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 553.345472] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 553.501996] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 559.457461] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 568.092310] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 573.886895] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 574.243441] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 579.856192] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 580.258084] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 584.161911] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 584.167082] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 584.169172] LustreError: Skipped 1 previous similar message [ 584.173426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 584.188810] Lustre: Skipped 1 previous similar message [ 584.208145] Lustre: Skipped 1 previous similar message [ 589.066683] Lustre: server umount lustre-OST0000 complete [ 593.889631] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 593.900085] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 593.910269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 600.170282] Lustre: server umount lustre-MDT0000 complete [ 604.649788] LustreError: 15009:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 604.672500] LustreError: 15009:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 605.682713] LustreError: 14994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776994814 with bad export cookie 17375300153031225677 [ 605.685238] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 605.690583] LustreError: 14994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 606.038338] Lustre: server umount lustre-MDT0001 complete [ 614.006773] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 21:40:20 (1776994820) [ 622.716898] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 623.593620] LustreError: 18861:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 623.703852] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 628.706765] LustreError: 18862:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 628.927639] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 638.840152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 638.953301] LustreError: 18861:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 638.981874] LustreError: 18861:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 644.891898] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 652.300833] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 657.842619] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 666.634857] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 674.741288] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 681.670978] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 688.990376] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 700.385163] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 700.390825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 700.402211] Lustre: Skipped 3 previous similar messages [ 700.406524] Lustre: Skipped 5 previous similar messages [ 703.862849] Lustre: server umount lustre-MDT0000 complete [ 705.512273] LustreError: 21015:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 705.537738] LustreError: 21015:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 708.899186] LustreError: 18842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776994918 with bad export cookie 17375300153031226559 [ 708.915328] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 709.310163] Lustre: server umount lustre-MDT0001 complete [ 725.073473] Lustre: server umount lustre-OST0000 complete [ 749.365438] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 760.796220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 761.251257] LustreError: 22895:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 761.363786] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 761.375522] Lustre: Skipped 2 previous similar messages [ 766.225533] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 775.785360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 782.006243] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 785.743482] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 793.511408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 802.086132] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 812.246402] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 812.454682] Lustre: lustre-OST0001: new disk, initializing [ 812.459779] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 813.786214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 813.799497] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 813.954237] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 818.933141] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 829.791954] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 834.350617] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 852.963376] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 852.985230] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 852.996285] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 853.000780] Lustre: Skipped 1 previous similar message [ 858.442514] Lustre: server umount lustre-MDT0000 complete [ 860.136706] LustreError: 22896:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 860.154770] LustreError: 22896:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 6 previous similar messages [ 862.755865] LustreError: 22876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776995071 with bad export cookie 17375300153031227406 [ 862.758734] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 862.765386] LustreError: 22876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 863.044798] Lustre: server umount lustre-MDT0001 complete [ 877.819853] Lustre: server umount lustre-OST0000 complete [ 881.770526] Lustre: 3669:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776995074/real 1776995074] req@ffff9b854568f480 x1863313623612032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1776995090 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 881.793652] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 881.805929] Lustre: Skipped 3 previous similar messages [ 882.328803] Lustre: server umount lustre-OST0001 complete [ 899.280414] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 910.880969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 911.726693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 911.739360] Lustre: Skipped 3 previous similar messages [ 916.918763] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 925.856679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 930.956888] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 934.898690] Lustre: 29260:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 943.002607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 951.045170] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 961.072717] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 968.299368] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 975.771431] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 980.429369] Lustre: 31103:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 992.740805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 992.769649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 992.799380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 992.805903] Lustre: Skipped 4 previous similar messages [ 999.298840] Lustre: server umount lustre-MDT0000 complete [ 1002.486109] LustreError: 28146:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1002.504249] LustreError: 28146:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 11 previous similar messages [ 1004.253324] LustreError: 28128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776995213 with bad export cookie 17375300153031228701 [ 1004.257404] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1004.298940] LustreError: 28128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1004.825992] Lustre: server umount lustre-MDT0001 complete [ 1019.493643] Lustre: server umount lustre-OST0000 complete [ 1023.468587] Lustre: 3670:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776995216/real 1776995216] req@ffff9b857c078380 x1863313623666688/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1776995232 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1024.748953] Lustre: server umount lustre-OST0001 complete [ 1035.807457] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 21:47:22 (1776995242) [ 1065.376060] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1076.557335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1082.327203] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1091.524177] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1096.314630] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1099.710848] Lustre: 35009:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1106.690905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1113.958141] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1123.116153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1130.291794] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1137.952554] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1142.022495] Lustre: 36854:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1153.509683] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1153.518786] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1153.532085] Lustre: Skipped 4 previous similar messages [ 1153.537536] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1153.539737] Lustre: Skipped 4 previous similar messages [ 1158.382501] Lustre: server umount lustre-MDT0000 complete [ 1163.026641] LustreError: 33872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776995372 with bad export cookie 17375300153031229807 [ 1163.035889] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1163.037348] LustreError: 33872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1163.452882] Lustre: server umount lustre-MDT0001 complete [ 1178.266471] Lustre: server umount lustre-OST0000 complete [ 1179.936249] Lustre: 3668:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776995373/real 1776995373] req@ffff9b8546e90700 x1863313623718784/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1776995389 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1182.278211] Lustre: server umount lustre-OST0001 complete [ 1191.829671] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 21:49:58 (1776995398) [ 1208.825149] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1220.159582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1220.892694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1220.907085] Lustre: Skipped 7 previous similar messages [ 1225.912240] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1235.454204] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1241.021980] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1243.886746] Lustre: 40091:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1250.546123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1257.176959] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1261.101481] LustreError: 40447:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1261.126375] LustreError: 40447:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 14 previous similar messages [ 1265.872486] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1272.205223] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1279.759723] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1282.805302] Lustre: 41934:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1288.731374] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1288.741331] Lustre: Skipped 1 previous similar message [ 1291.749817] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1291.751455] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1291.771264] Lustre: Skipped 6 previous similar messages [ 1291.791443] Lustre: Skipped 7 previous similar messages [ 1297.713250] Lustre: server umount lustre-MDT0000 complete [ 1301.577769] LustreError: 38959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776995510 with bad export cookie 17375300153031230913 [ 1301.584654] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1301.590877] LustreError: 38959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1308.108037] Lustre: server umount lustre-MDT0001 complete [ 1316.443368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1331.197974] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1336.177458] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1340.316625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1346.039968] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1346.043700] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1346.049667] Lustre: Skipped 1 previous similar message [ 1351.761766] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1353.392846] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1357.152546] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1359.045099] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1366.514322] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1371.391618] Lustre: server umount lustre-MDT0000 complete [ 1375.284460] LustreError: 38957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776995584 with bad export cookie 17375300153031232103 [ 1375.288227] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1375.296408] LustreError: 38957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1375.785600] Lustre: server umount lustre-MDT0001 complete [ 1389.943326] Lustre: server umount lustre-OST0000 complete [ 1392.097322] Lustre: 3670:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776995585/real 1776995585] req@ffff9b8441ec5500 x1863313623806464/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1776995601 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1393.559263] Lustre: server umount lustre-OST0001 complete [ 1402.090582] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 21:53:29 (1776995609) [ 1419.016614] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1429.334087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1434.806211] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1443.120656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1448.176526] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1451.247709] Lustre: 47424:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1458.158048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1464.787282] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1474.012116] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1481.589062] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1489.273079] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1499.306855] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1511.753231] Lustre: server umount lustre-MDT0000 complete [ 1516.142847] LustreError: 46286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776995725 with bad export cookie 17375300153031232887 [ 1516.145590] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1516.152396] LustreError: 46286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1516.553983] Lustre: server umount lustre-MDT0001 complete [ 1531.891174] Lustre: server umount lustre-OST0000 complete [ 1532.705339] Lustre: 3670:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776995725/real 1776995725] req@ffff9b854fb35180 x1863313623868160/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1776995741 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1535.673451] Lustre: server umount lustre-OST0001 complete [ 1545.390502] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 21:55:52 (1776995752) [ 1547.875508] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1550.057386] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 21:55:57 (1776995757) [ 1561.806259] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 1572.007059] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1593.813207] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1594.145624] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1594.242786] Lustre: lustre-MDT0000: new disk, initializing [ 1594.314133] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1598.066254] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1604.988793] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1611.404581] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1611.715511] Lustre: lustre-OST0000: new disk, initializing [ 1611.722108] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1611.728535] Lustre: Skipped 1 previous similar message [ 1613.579426] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1613.585144] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1613.610391] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1617.159601] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1624.635311] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1626.727519] Lustre: DEBUG MARKER: create test files [ 1665.009505] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1665.029989] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1665.044575] Lustre: Skipped 13 previous similar messages [ 1665.051930] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1665.054818] Lustre: Skipped 18 previous similar messages [ 1666.836513] Lustre: server umount lustre-OST0000 complete [ 1671.237331] Lustre: server umount lustre-MDT0000 complete [ 1682.747539] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1685.265601] Key type lgssc unregistered [ 1685.556090] LNet: 54824:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1685.562872] LNetError: 54824:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1685.581848] LNet: Removed LNI 192.168.202.143@tcp [ 1686.426153] Key type .llcrypt unregistered [ 1686.427664] Key type ._llcrypt unregistered [ 1718.926693] Key type ._llcrypt registered [ 1718.941844] Key type .llcrypt registered [ 1719.091726] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1720.574373] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1720.616056] alg: No test for adler32 (adler32-zlib) [ 1721.671366] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 1721.869267] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1723.504580] Key type lgssc registered [ 1724.936694] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1733.395426] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1733.427205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1735.223629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1739.997729] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1746.567875] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1753.838936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1754.264238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1760.434715] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1760.554541] Lustre: 57428:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1760.561384] Lustre: 57428:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1760.567176] Lustre: 57428:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1760.572824] Lustre: 57428:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1760.581197] Lustre: 57428:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1760.593824] Lustre: 57428:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1761.083619] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1768.316737] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1774.212738] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1781.009493] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1791.458176] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1791.469581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1791.497971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1795.803197] Lustre: server umount lustre-OST0000 complete [ 1800.560395] Lustre: server umount lustre-MDT0000 complete [ 1815.088304] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1818.159054] Key type lgssc unregistered [ 1818.513062] LNet: 58907:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1818.523439] LNetError: 58907:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1818.539532] LNet: Removed LNI 192.168.202.143@tcp [ 1819.677952] Key type .llcrypt unregistered [ 1819.680176] Key type ._llcrypt unregistered [ 1853.151698] Key type ._llcrypt registered [ 1853.159482] Key type .llcrypt registered [ 1853.290897] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1854.885420] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1854.937477] alg: No test for adler32 (adler32-zlib) [ 1855.952288] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 1856.182153] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1857.960199] Key type lgssc registered [ 1859.438181] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1867.277501] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1867.298660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1868.782858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1873.684678] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1879.183116] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1886.664737] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1887.072138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1893.247870] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1893.405655] Lustre: 61627:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1893.423972] Lustre: 61627:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1893.431017] Lustre: 61627:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1893.439046] Lustre: 61627:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1893.444575] Lustre: 61627:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1893.451932] Lustre: 61627:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1894.184905] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1900.595392] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1907.152694] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1918.953784] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1918.956533] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1918.968575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1923.021978] Lustre: server umount lustre-OST0000 complete [ 1927.869529] Lustre: server umount lustre-MDT0000 complete [ 1940.766985] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1944.071461] Key type lgssc unregistered [ 1944.323480] LNet: 63054:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1944.331484] LNetError: 63054:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1944.363941] LNet: Removed LNI 192.168.202.143@tcp [ 1945.527786] Key type .llcrypt unregistered [ 1945.534420] Key type ._llcrypt unregistered [ 1968.531979] Key type ._llcrypt registered [ 1968.535519] Key type .llcrypt registered [ 1968.643393] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 1987.192420] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1988.342599] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1988.361247] alg: No test for adler32 (adler32-zlib) [ 1989.481547] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 1989.644649] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1991.336187] Key type lgssc registered [ 1992.667070] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2027.168780] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2027.182534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2028.412777] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2028.445751] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2028.525908] Lustre: lustre-MDT0000: new disk, initializing [ 2028.605351] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2028.626413] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2032.998955] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2044.217652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2044.329699] Lustre: 66178:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2044.444915] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2044.447821] Lustre: Skipped 1 previous similar message [ 2044.563255] Lustre: lustre-MDT0001: new disk, initializing [ 2044.692358] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2044.728785] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2044.743758] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2049.629601] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2056.716868] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2060.454447] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2067.786554] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2068.140388] Lustre: lustre-OST0000: new disk, initializing [ 2068.144182] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2068.215785] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2073.644908] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2077.796858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2077.818773] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2077.945658] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2081.240619] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2086.532748] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2086.690786] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2091.428819] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2091.683664] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2098.669675] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2098.684507] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2098.695077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2103.782090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2103.803042] Lustre: Skipped 2 previous similar messages [ 2108.385204] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2108.623062] Lustre: server umount lustre-OST0000 complete [ 2114.018171] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2114.021128] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2114.028357] Lustre: Skipped 2 previous similar messages [ 2114.035131] Lustre: Skipped 1 previous similar message [ 2119.223798] Lustre: server umount lustre-MDT0000 complete [ 2123.332990] LustreError: 67919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776996332 with bad export cookie 17254810242194711508 [ 2123.339057] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2123.632169] Lustre: server umount lustre-MDT0001 complete [ 2134.130633] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:05:40 (1776996340) [ 2145.993384] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2192.619089] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 2204.581211] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2247.301283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2247.521220] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2247.559662] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2247.641948] Lustre: lustre-MDT0000: new disk, initializing [ 2247.725293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2247.744636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2251.674819] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2262.955679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2263.055441] Lustre: 73745:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2263.104686] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2263.109177] Lustre: Skipped 1 previous similar message [ 2263.195994] Lustre: lustre-MDT0001: new disk, initializing [ 2263.296897] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2263.325429] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2263.329204] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2268.551876] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2276.727601] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2281.307914] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2288.792985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2289.052298] Lustre: lustre-OST0000: new disk, initializing [ 2289.055971] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2289.116068] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2295.521537] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2298.964346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2298.992387] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2299.110035] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2303.024777] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2307.084045] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2307.268446] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2310.914257] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2311.023969] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2314.723839] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2314.724917] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2314.739449] Lustre: Skipped 1 previous similar message [ 2314.744466] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2314.744478] LustreError: Skipped 1 previous similar message [ 2319.065730] Lustre: server umount lustre-OST0000 complete [ 2324.971738] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2324.981842] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2324.991266] LustreError: Skipped 1 previous similar message [ 2324.992625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2324.992633] Lustre: Skipped 1 previous similar message [ 2325.056845] Lustre: Skipped 1 previous similar message [ 2330.435961] Lustre: server umount lustre-MDT0000 complete [ 2334.602886] LustreError: 73737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776996543 with bad export cookie 17254810242194712390 [ 2334.606343] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2334.619619] LustreError: 73737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2335.064480] Lustre: server umount lustre-MDT0001 complete [ 2343.540355] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:09:10 (1776996550) [ 2350.984626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2351.279506] LustreError: 77752:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2351.290337] LustreError: 77752:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 2351.361327] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2355.485643] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2356.712392] LustreError: 77753:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2361.827229] LustreError: 77752:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2364.655174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2369.712406] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2375.569661] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2379.718675] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2386.991409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2387.316358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2387.325500] Lustre: Skipped 1 previous similar message [ 2393.440613] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2398.849043] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2403.014204] Lustre: 80083:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting d2a07d7a-ee8b-404c-954d-72b2fbeb0049 at adminstrative request [ 2407.932465] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2407.942045] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2418.150678] Lustre: MGS: Client d2a07d7a-ee8b-404c-954d-72b2fbeb0049 (at 0@lo) reconnecting [ 2418.163690] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xef75667ed0c9aab8 to 0xef75667ed0c9ad58 [ 2418.199409] Lustre: MGC192.168.202.143@tcp: Connection restored to 0@lo (at 0@lo) [ 2440.988911] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2441.201014] Lustre: lustre-OST0001: new disk, initializing [ 2441.213660] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2447.181793] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2447.527788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2447.537385] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2447.626792] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2455.140886] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2463.201975] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2463.208289] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2463.220922] Lustre: Skipped 1 previous similar message [ 2463.225545] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2463.229441] Lustre: Skipped 4 previous similar messages [ 2464.327171] Lustre: server umount lustre-OST0001 complete [ 2468.324405] LustreError: 79344:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2468.337584] LustreError: 79344:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 2473.457953] LustreError: 79347:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2473.478067] LustreError: 79347:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 2474.502397] Lustre: server umount lustre-OST0000 complete [ 2483.180664] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2483.194397] LustreError: Skipped 1 previous similar message [ 2483.199657] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2483.223847] Lustre: Skipped 2 previous similar messages [ 2483.233754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2483.240424] Lustre: Skipped 4 previous similar messages [ 2485.262527] Lustre: server umount lustre-MDT0000 complete [ 2488.801288] LustreError: 81033:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2488.822563] LustreError: 81033:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 2488.972747] LustreError: 80473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776996698 with bad export cookie 17254810242194713944 [ 2488.974202] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2488.981232] LustreError: 80473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2489.249722] Lustre: server umount lustre-MDT0001 complete [ 2498.655740] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:11:45 (1776996705) [ 2500.351764] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2502.075622] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:11:49 (1776996709) [ 2503.912514] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2505.848311] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:11:52 (1776996712) [ 2507.976055] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2510.618644] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:11:57 (1776996717) [ 2513.561716] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2515.622199] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:12:02 (1776996722) [ 2517.463963] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2525.960034] loop: module loaded [ 2525.988532] loop0: detected capacity change from 0 to 4294967296 [ 2526.109589] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2528.493028] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2533.942757] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2538.586993] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2568.260506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2568.295196] LustreError: 84474:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2568.299426] LustreError: 84474:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2568.309915] LustreError: 84474:0:(tgt_mount.c:2532:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2568.317380] LustreError: 84474:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2591.496627] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 2603.316395] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2645.429809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2645.728843] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2645.755277] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2645.876447] Lustre: lustre-MDT0000: new disk, initializing [ 2646.026264] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2646.031719] Lustre: Skipped 1 previous similar message [ 2646.044306] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2649.354894] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2663.279853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2663.504168] Lustre: 87306:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2663.904877] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2670.054916] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2679.137570] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2683.637392] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2691.314501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2697.318250] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2697.421530] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2698.560095] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2707.184417] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2711.844937] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2712.037786] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2716.892405] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2717.016572] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2723.301211] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2723.312087] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2723.370749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2723.372232] Lustre: Skipped 2 previous similar messages [ 2723.414385] Lustre: Skipped 2 previous similar messages [ 2724.933950] Lustre: server umount lustre-OST0000 complete [ 2744.288140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2744.600877] Lustre: server umount lustre-MDT0000 complete [ 2748.523566] LustreError: 87299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776996957 with bad export cookie 17254810242194714560 [ 2748.532857] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2749.181866] Lustre: server umount lustre-MDT0001 complete [ 2758.320851] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:16:05 (1776996965) [ 2766.306701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2767.179421] LustreError: 91317:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2772.731420] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2782.739721] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2783.398389] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2783.403115] Lustre: Skipped 3 previous similar messages [ 2789.097967] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2796.224584] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2802.097248] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2809.555509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2816.237704] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2823.365479] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2829.243468] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2829.528103] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2834.783119] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2835.034264] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2842.503959] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2842.695722] Lustre: lustre-OST0001: new disk, initializing [ 2842.698738] Lustre: Skipped 2 previous similar messages [ 2842.706229] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2842.713017] Lustre: Skipped 3 previous similar messages [ 2848.938269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2848.957987] Lustre: Skipped 2 previous similar messages [ 2848.968385] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2849.078698] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2850.009579] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2860.552550] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2866.166701] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2866.395873] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2871.807280] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2872.145301] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2893.550585] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2923.068893] Lustre: Failing over lustre-MDT0000 [ 2923.682820] Lustre: server umount lustre-MDT0000 complete [ 2926.013419] LustreError: 92831:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2926.053073] LustreError: 92831:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 2926.060644] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2926.111347] Lustre: Skipped 4 previous similar messages [ 2942.304212] Lustre: 64398:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776997135/real 1776997135] req@000000005c2222ae x1863315565587456/t0(0) o400->MGC192.168.202.143@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1776997151 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2942.359675] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2952.068070] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2952.072449] LDISKFS-fs (dm-0): recovery complete [ 2952.079538] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2953.152796] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2953.632970] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2953.645072] Lustre: 91313:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2958.318493] Lustre: 92029:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2958.340677] Lustre: 92029:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 1 previous similar message [ 2958.374309] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2958.468689] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2958.801116] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2959.126228] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2959.138808] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 1 previous similar message [ 2959.258449] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2959.468922] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2959.471953] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 1 previous similar message [ 2959.484156] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2959.520844] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 7 previous similar messages [ 2960.172390] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2960.185191] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 2 previous similar messages [ 2960.504075] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2960.514456] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 2 previous similar messages [ 2961.798206] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2961.810180] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 6 previous similar messages [ 2962.469533] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2962.486963] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 6 previous similar messages [ 2962.808109] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2962.812155] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 6 previous similar messages [ 2966.009634] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2966.021889] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 12 previous similar messages [ 2966.661223] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2966.674126] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 12 previous similar messages [ 2966.992246] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2966.998494] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 12 previous similar messages [ 2974.321754] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2974.340933] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 25 previous similar messages [ 2974.666513] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2974.684090] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 24 previous similar messages [ 2975.010174] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2975.012555] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 24 previous similar messages [ 2990.636154] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2990.662105] Lustre: 97016:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 50 previous similar messages [ 2990.672965] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2990.684641] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 49 previous similar messages [ 2991.329937] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2991.342283] LustreError: 97016:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 50 previous similar messages [ 3013.059533] Lustre: 97016:0:(ldlm_lib.c:2379:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3013.079141] Lustre: 97016:0:(ldlm_lib.c:2389:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3013.096825] Lustre: 97016:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client f989352e-4af6-450b-8c46-732f1d63045b@192.168.202.43@tcp [ 3013.115779] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3013.127062] Lustre: 97016:0:(ldlm_lib.c:1899:abort_req_replay_queue()) @@@ aborted: req@00000000614ec903 x1863315546508032/t0(8589934767) o101->f989352e-4af6-450b-8c46-732f1d63045b@192.168.202.43@tcp:542/0 lens 592/0 e 0 to 0 dl 1776997232 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3013.159221] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3013.171601] Lustre: lustre-MDT0000: Denying connection for new client f989352e-4af6-450b-8c46-732f1d63045b (at 192.168.202.43@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3013.187864] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3013.227328] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3013.237216] Lustre: Skipped 2 previous similar messages [ 3013.238240] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3013.287160] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 3013.289516] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3033.579055] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3033.603636] LustreError: Skipped 2 previous similar messages [ 3033.605791] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3033.631818] Lustre: Skipped 1 previous similar message [ 3033.643720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3033.663209] Lustre: Skipped 7 previous similar messages [ 3045.858330] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3046.018875] Lustre: server umount lustre-OST0000 complete [ 3050.478056] LustreError: 94603:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3050.511891] LustreError: 94603:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 41 previous similar messages [ 3065.312171] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3065.821540] Lustre: server umount lustre-OST0001 complete [ 3076.925746] Lustre: server umount lustre-MDT0000 complete [ 3080.668692] LustreError: 91298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776997289 with bad export cookie 17254810242194765772 [ 3080.685881] LustreError: 91298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3080.692363] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3081.004896] Lustre: server umount lustre-MDT0001 complete [ 3090.270390] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 22:21:37 (1776997297) [ 3098.399903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3098.408775] Lustre: *** cfs_fail_loc=197, val=0*** [ 3114.016653] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3119.137181] LustreError: 98980:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 3124.303914] Lustre: *** cfs_fail_loc=197, val=0*** [ 3124.305307] Lustre: Skipped 65 previous similar messages [ 3144.800629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3144.811290] Lustre: Skipped 3 previous similar messages [ 3152.191312] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3158.628077] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3161.057752] Lustre: server umount lustre-OST0000 complete [ 3170.711820] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:22:57 (1776997377) [ 3195.815958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3196.550171] LustreError: 100493:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3196.580422] LustreError: 100493:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 4 previous similar messages [ 3202.577246] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3213.257761] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3218.212271] LustreError: 100524:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3218.252425] LustreError: 100524:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3218.262672] LustreError: 100524:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 3218.673956] Lustre: server umount lustre-MDT0000 complete [ 3256.994367] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 3269.438867] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 3314.307577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3314.621220] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3314.682461] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3314.798943] Lustre: lustre-MDT0000: new disk, initializing [ 3314.951065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3320.206216] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3333.050979] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3333.248550] Lustre: 104554:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3333.674139] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3338.993457] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3347.662623] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3352.609677] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3361.076137] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3367.709177] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3368.718261] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3378.730145] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3384.099477] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3384.344306] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3391.264689] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3391.696320] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3398.625182] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3398.637669] LustreError: Skipped 3 previous similar messages [ 3398.643373] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3398.655717] Lustre: Skipped 5 previous similar messages [ 3398.665966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3398.682244] Lustre: Skipped 18 previous similar messages [ 3400.789903] Lustre: server umount lustre-OST0000 complete [ 3410.881110] Lustre: server umount lustre-MDT0000 complete [ 3415.427256] LustreError: 104546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776997624 with bad export cookie 17254810242194776328 [ 3415.441729] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3415.443365] LustreError: 104546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3415.946044] Lustre: server umount lustre-MDT0001 complete [ 3426.043692] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 22:27:13 (1776997633) [ 3436.848384] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 3439.894228] Key type lgssc unregistered [ 3440.252246] LNet: 108620:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3440.266363] LNetError: 108620:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3440.289828] LNet: Removed LNI 192.168.202.143@tcp [ 3441.663363] Key type .llcrypt unregistered [ 3441.664759] Key type ._llcrypt unregistered [ 3466.337143] Key type ._llcrypt registered [ 3466.340032] Key type .llcrypt registered [ 3466.429110] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 3484.264491] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 3485.773709] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3485.840835] alg: No test for adler32 (adler32-zlib) [ 3486.885452] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 3487.084497] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 3488.768557] Key type lgssc registered [ 3490.348441] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3542.438205] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3542.462464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3544.236836] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3544.337388] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3544.505831] Lustre: lustre-MDT0000: new disk, initializing [ 3544.751641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3544.856434] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3550.241249] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3562.347191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3562.832182] Lustre: lustre-OST0000: new disk, initializing [ 3562.836170] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3562.842630] Lustre: Skipped 1 previous similar message [ 3562.947381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3568.788173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3568.807235] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3568.857184] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3570.796816] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3581.905920] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3582.111638] Lustre: lustre-OST0001: new disk, initializing [ 3582.116141] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3582.219643] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3588.749376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3588.762401] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3588.798179] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3589.135490] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3603.951821] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3603.972461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3603.978598] Lustre: Skipped 1 previous similar message [ 3605.623295] Lustre: server umount lustre-MDT0000 complete [ 3612.798827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3618.097812] LustreError: 111947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776997827 with bad export cookie 15177300483611486150 [ 3618.105575] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3618.113335] LustreError: 111947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3618.269870] Lustre: server umount lustre-OST0000 complete [ 3622.594121] Lustre: server umount lustre-OST0001 complete [ 3632.983173] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 22:30:40 (1776997840) [ 3643.969357] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 3673.212820] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 22:31:19 (1776997879) [ 3722.531803] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 3735.769267] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 3784.130035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3784.608570] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3784.649257] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3784.759302] Lustre: lustre-MDT0000: new disk, initializing [ 3784.906126] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3784.955214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3790.801905] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3804.956027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3805.106124] Lustre: 120284:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3805.170609] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3805.180361] Lustre: Skipped 1 previous similar message [ 3805.279974] Lustre: lustre-MDT0001: new disk, initializing [ 3805.386729] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3805.458790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3805.493202] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3810.599815] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3821.017156] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3826.934873] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3836.159347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3836.544338] Lustre: lustre-OST0000: new disk, initializing [ 3836.548475] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3836.678583] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3844.210601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3844.227527] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3844.333411] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3844.431570] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3853.709706] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3860.272906] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3860.529349] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3866.714590] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3866.981432] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3870.177523] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3870.187457] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3870.217955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3870.227202] Lustre: Skipped 2 previous similar messages [ 3870.255551] Lustre: Skipped 1 previous similar message [ 3875.331692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3875.591168] Lustre: server umount lustre-OST0000 complete [ 3882.465226] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3882.480108] LustreError: Skipped 1 previous similar message [ 3882.481665] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3882.518481] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3882.527186] Lustre: Skipped 1 previous similar message [ 3885.538116] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3886.943547] Lustre: server umount lustre-MDT0000 complete [ 3891.311668] LustreError: 120278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776998100 with bad export cookie 15177300483611487375 [ 3891.319884] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3891.335835] LustreError: 120278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3891.789949] Lustre: server umount lustre-MDT0001 complete [ 3903.483806] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 22:35:10 (1776998110) [ 3911.846774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3912.638890] LustreError: 124296:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3912.666686] LustreError: 124296:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 3912.775166] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3917.802538] LustreError: 124297:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3918.725592] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3921.900299] LustreError: 124296:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3927.010436] LustreError: 124297:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3930.862774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3937.589437] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3944.165648] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3949.812106] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3958.239295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3958.526909] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3958.531328] Lustre: Skipped 1 previous similar message [ 3965.014889] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3973.740942] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3989.721709] Lustre: *** cfs_fail_loc=159, val=0*** [ 4006.363101] Lustre: lustre-MDT0000: Client 27b9437e-85b9-43a8-8592-ec2f6f258a42 (at 192.168.202.43@tcp) reconnecting [ 4008.018876] Lustre: *** cfs_fail_loc=159, val=0*** [ 4008.020325] Lustre: Skipped 5 previous similar messages [ 4024.287081] Lustre: lustre-MDT0000: Client 27b9437e-85b9-43a8-8592-ec2f6f258a42 (at 192.168.202.43@tcp) reconnecting [ 4031.797773] Lustre: server umount lustre-OST0000 complete [ 4038.634399] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4038.644932] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4038.651893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4038.651902] Lustre: Skipped 2 previous similar messages [ 4046.937049] Lustre: server umount lustre-MDT0000 complete [ 4047.841095] Lustre: 109966:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776998241/real 1776998241] req@00000000b9d7954e x1863317136195200/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1776998257 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4047.889229] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4047.919699] Lustre: Skipped 1 previous similar message [ 4047.928844] LustreError: 124296:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4047.951943] LustreError: 124296:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4048.934790] Lustre: 109968:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776998242/real 1776998242] req@00000000d7edccc7 x1863317136195968/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1776998258 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4051.161519] LustreError: 124277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776998260 with bad export cookie 15177300483611488250 [ 4051.162952] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4051.177613] LustreError: 124277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4051.562425] Lustre: server umount lustre-MDT0001 complete [ 4063.905322] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4067.074796] Key type lgssc unregistered [ 4067.339050] LNet: 127735:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4067.345522] LNetError: 127735:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4067.362159] LNet: Removed LNI 192.168.202.143@tcp [ 4068.374536] Key type .llcrypt unregistered [ 4068.377489] Key type ._llcrypt unregistered [ 4091.815313] Key type ._llcrypt registered [ 4091.817743] Key type .llcrypt registered [ 4091.961208] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4092.778721] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4092.813991] alg: No test for adler32 (adler32-zlib) [ 4093.773325] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 4093.879692] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4095.480161] Key type lgssc registered [ 4096.394192] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4105.921108] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4105.933378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4107.981595] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4115.296860] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4118.514881] LustreError: 129130:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4118.556699] LustreError: 129130:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4123.643052] LustreError: 129131:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4127.122815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4127.919317] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4133.517441] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4140.430232] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4146.013802] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4154.482613] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4155.003506] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4162.851985] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4169.145850] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4178.566329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4189.655603] Lustre: *** cfs_fail_loc=159, val=0*** [ 4206.026350] Lustre: lustre-MDT0000: Client a9d98494-cbaf-4795-9008-3634c7d558d2 (at 192.168.202.43@tcp) reconnecting [ 4209.293791] Lustre: *** cfs_fail_loc=159, val=0*** [ 4224.453216] Lustre: lustre-MDT0001: Client a9d98494-cbaf-4795-9008-3634c7d558d2 (at 192.168.202.43@tcp) reconnecting [ 4226.135443] Lustre: *** cfs_fail_loc=159, val=0*** [ 4226.156934] Lustre: Skipped 3 previous similar messages [ 4242.384107] Lustre: lustre-MDT0001: Client a9d98494-cbaf-4795-9008-3634c7d558d2 (at 192.168.202.43@tcp) reconnecting [ 4247.129054] Lustre: 131849:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4249.939736] Lustre: *** cfs_fail_loc=159, val=0*** [ 4249.946197] Lustre: Skipped 4 previous similar messages [ 4265.424985] Lustre: lustre-MDT0000: Client 64316d0f-cad5-405d-9038-8974af246030 (at 192.168.202.43@tcp) reconnecting [ 4267.281594] Lustre: *** cfs_fail_loc=159, val=0*** [ 4267.283335] Lustre: Skipped 12 previous similar messages [ 4282.848478] Lustre: lustre-MDT0000: Client 64316d0f-cad5-405d-9038-8974af246030 (at 192.168.202.43@tcp) reconnecting [ 4284.861762] Lustre: 132143:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4291.566197] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4291.576712] Lustre: Skipped 1 previous similar message [ 4291.578276] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4291.578307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4296.685790] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4296.698024] Lustre: Skipped 2 previous similar messages [ 4297.383214] Lustre: server umount lustre-OST0000 complete [ 4301.796101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4301.800374] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4301.812713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4301.839217] Lustre: Skipped 1 previous similar message [ 4304.008392] Lustre: server umount lustre-MDT0000 complete [ 4306.921488] LustreError: 131283:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4306.945561] LustreError: 131283:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4308.894442] LustreError: 129111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776998518 with bad export cookie 6870154308574792491 [ 4308.915673] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4308.918225] LustreError: 129111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4309.481661] Lustre: server umount lustre-MDT0001 complete [ 4322.862933] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4326.358079] Key type lgssc unregistered [ 4326.704383] LNet: 133178:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4326.710497] LNetError: 133178:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4326.745504] LNet: Removed LNI 192.168.202.143@tcp [ 4327.881414] Key type .llcrypt unregistered [ 4327.884059] Key type ._llcrypt unregistered [ 4359.176953] Key type ._llcrypt registered [ 4359.178371] Key type .llcrypt registered [ 4359.353428] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4361.877375] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4361.904882] alg: No test for adler32 (adler32-zlib) [ 4362.960419] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 4363.278112] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4365.096253] Key type lgssc registered [ 4366.778901] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4377.923902] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4377.936889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4379.414910] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4385.144888] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4388.835542] LustreError: 134579:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4388.855921] LustreError: 134579:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4393.952989] LustreError: 134580:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4396.334571] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4397.022828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4402.574967] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4409.678035] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4415.342686] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4423.214597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4423.651318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4424.714263] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4430.603892] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4436.803330] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4442.060508] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4446.637346] Lustre: 136824:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4450.764710] Lustre: 136872:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4452.847981] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4453.930201] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4458.460762] Lustre: 137019:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4459.572967] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4460.596573] Lustre: 137118:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4465.637480] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4465.646071] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4465.672876] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4467.687684] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4467.692915] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4467.693491] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4471.818060] Lustre: server umount lustre-OST0000 complete [ 4477.923228] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4477.929616] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4477.951557] Lustre: Skipped 1 previous similar message [ 4477.970661] Lustre: Skipped 2 previous similar messages [ 4482.886262] Lustre: server umount lustre-MDT0000 complete [ 4483.048932] LustreError: 134575:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4483.076641] LustreError: 134575:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4487.142245] LustreError: 134580:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4487.160847] LustreError: 134580:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4487.450682] LustreError: 134560:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776998696 with bad export cookie 6285941165372870628 [ 4487.458268] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4487.462823] LustreError: 134560:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4487.789769] Lustre: server umount lustre-MDT0001 complete [ 4503.577859] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4507.630417] Key type lgssc unregistered [ 4507.907321] LNet: 138163:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4507.918294] LNetError: 138163:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4508.981076] LNet: Removed LNI 192.168.202.143@tcp [ 4510.007804] Key type .llcrypt unregistered [ 4510.011382] Key type ._llcrypt unregistered [ 4542.398163] Key type ._llcrypt registered [ 4542.400648] Key type .llcrypt registered [ 4542.533353] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4543.933349] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4543.990279] alg: No test for adler32 (adler32-zlib) [ 4545.335308] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 4545.796097] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4547.544222] Key type lgssc registered [ 4549.100747] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4558.762134] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4558.797267] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4560.577744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4566.110298] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4570.081736] LustreError: 139578:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4570.106372] LustreError: 139578:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4575.210407] LustreError: 139579:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4576.508095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4577.068499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4582.430362] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4588.194939] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4592.332513] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4601.120833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4601.502119] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4602.558366] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4605.696591] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4609.960507] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4617.077276] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4624.590834] Lustre: *** cfs_fail_loc=159, val=0*** [ 4641.266926] Lustre: lustre-MDT0000: Client 15e65274-d092-46f0-a666-6fbab7355514 (at 192.168.202.43@tcp) reconnecting [ 4648.931531] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4648.946591] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4648.980878] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4652.002279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4652.016741] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4653.932136] Lustre: server umount lustre-OST0000 complete [ 4659.170667] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4659.180717] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4659.195641] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4659.206874] Lustre: Skipped 1 previous similar message [ 4662.243914] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4662.247499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4662.314495] Lustre: Skipped 1 previous similar message [ 4664.950156] Lustre: server umount lustre-MDT0000 complete [ 4667.370973] LustreError: 139574:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4667.398494] LustreError: 139574:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4669.474411] LustreError: 139559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776998878 with bad export cookie 15702864485207264138 [ 4669.483961] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4669.489795] LustreError: 139559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4670.060746] Lustre: server umount lustre-MDT0001 complete [ 4683.024979] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4686.824935] Key type lgssc unregistered [ 4687.294488] LNet: 142895:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4687.304860] LNetError: 142895:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4687.320368] LNet: Removed LNI 192.168.202.143@tcp [ 4688.756886] Key type .llcrypt unregistered [ 4688.764660] Key type ._llcrypt unregistered [ 4722.162269] Key type ._llcrypt registered [ 4722.167344] Key type .llcrypt registered [ 4722.370260] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4724.233013] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4724.412261] alg: No test for adler32 (adler32-zlib) [ 4725.612863] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 4726.046323] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4727.832653] Key type lgssc registered [ 4729.803529] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4740.783069] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4740.837451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4742.597871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4747.114880] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4751.844845] LustreError: 144297:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4751.862139] LustreError: 144297:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4756.499213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4757.021540] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4763.150231] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4770.773583] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4776.334857] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4784.978122] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4785.511327] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4787.606309] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4791.920681] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4798.128020] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4801.904526] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4825.812105] Lustre: lustre-MDT0001: evicting 019cf211-0c0f-4255-8106-657bfdc0bfa5 (at 192.168.202.43@tcp) by administrative request [ 4825.823824] Lustre: Skipped 1 previous similar message [ 4858.345974] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4858.354294] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4858.380489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4861.950393] Lustre: server umount lustre-OST0000 complete [ 4868.580135] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4868.589537] Lustre: Skipped 1 previous similar message [ 4868.593962] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4868.597431] Lustre: Skipped 1 previous similar message [ 4873.231992] Lustre: server umount lustre-MDT0000 complete [ 4873.697974] LustreError: 144293:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4873.723409] LustreError: 144293:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4878.628298] LustreError: 144278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776999087 with bad export cookie 3307032718561080240 [ 4878.631711] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4878.638883] LustreError: 144278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4879.149706] Lustre: server umount lustre-MDT0001 complete [ 4892.548890] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4896.391915] Key type lgssc unregistered [ 4896.819347] LNet: 148012:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4896.829456] LNetError: 148012:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4896.857512] LNet: Removed LNI 192.168.202.143@tcp [ 4897.852144] Key type .llcrypt unregistered [ 4897.853923] Key type ._llcrypt unregistered [ 4980.826170] Key type ._llcrypt registered [ 4980.829131] Key type .llcrypt registered [ 4981.002662] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4982.067977] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4982.222694] alg: No test for adler32 (adler32-zlib) [ 4983.538481] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 4983.863310] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4985.624180] Key type lgssc registered [ 4987.480097] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4997.676674] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4997.693217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4999.665437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5005.349537] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5008.866947] LustreError: 150488:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5008.900823] LustreError: 150488:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 5013.987768] LustreError: 150487:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5014.916059] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5015.556690] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5021.277409] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5027.748612] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5031.676974] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5039.284498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5039.812623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5040.832155] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5043.941758] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5048.088538] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5054.694465] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5066.723252] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5066.748971] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5066.781133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5069.795185] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5069.809266] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5069.815243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5069.839281] Lustre: Skipped 1 previous similar message [ 5071.141847] Lustre: server umount lustre-OST0000 complete [ 5076.963640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5076.981833] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5077.011499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5080.035602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5080.057735] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5085.158732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5085.166367] Lustre: Skipped 2 previous similar messages [ 5090.274659] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5090.455683] Lustre: server umount lustre-MDT0000 complete [ 5095.393873] LustreError: 150487:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5095.413246] LustreError: 150487:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 5095.712861] LustreError: 150467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776999304 with bad export cookie 11960736512265015731 [ 5095.728753] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5095.735279] LustreError: 150467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5096.296921] Lustre: server umount lustre-MDT0001 complete [ 5109.817586] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5113.066806] Key type lgssc unregistered [ 5113.455726] LNet: 153674:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5113.459319] LNetError: 153674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5113.488942] LNet: Removed LNI 192.168.202.143@tcp [ 5114.636461] Key type .llcrypt unregistered [ 5114.639763] Key type ._llcrypt unregistered [ 5140.284982] Key type ._llcrypt registered [ 5140.291565] Key type .llcrypt registered [ 5140.437444] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5141.764464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5141.807636] alg: No test for adler32 (adler32-zlib) [ 5143.224247] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 5143.617486] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5145.457843] Key type lgssc registered [ 5147.361699] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5165.373740] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:56:12 (1776999372) [ 5185.250290] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 5195.429707] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5235.185333] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5235.217878] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5236.465691] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5236.523080] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5236.676465] Lustre: lustre-MDT0000: new disk, initializing [ 5236.842389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5236.877928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5242.278511] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5255.068919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5255.278686] Lustre: 157888:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5255.424640] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5255.427702] Lustre: Skipped 1 previous similar message [ 5255.550218] Lustre: lustre-MDT0001: new disk, initializing [ 5255.658994] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5255.707165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5255.714741] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5260.885537] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5269.868947] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5273.912625] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5281.469141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5281.781769] Lustre: lustre-OST0000: new disk, initializing [ 5281.786384] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5281.874255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5286.946887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5286.953666] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5287.087561] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5288.451344] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5295.746344] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5307.874666] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5307.881039] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5307.888304] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5307.893656] Lustre: Skipped 1 previous similar message [ 5312.506928] Lustre: server umount lustre-OST0000 complete [ 5316.072491] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5316.083421] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5316.089333] Lustre: Skipped 1 previous similar message [ 5316.101570] Lustre: Skipped 2 previous similar messages [ 5317.092173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5317.098291] Lustre: Skipped 1 previous similar message [ 5322.213550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5322.225520] Lustre: Skipped 1 previous similar message [ 5327.336552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5327.344602] Lustre: Skipped 1 previous similar message [ 5330.400139] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5330.602658] Lustre: server umount lustre-MDT0000 complete [ 5334.340542] LustreError: 157879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1776999543 with bad export cookie 16312340485630660609 [ 5334.345520] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5334.346972] LustreError: 157879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5334.616729] Lustre: server umount lustre-MDT0001 complete [ 5344.846137] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5347.710742] Key type lgssc unregistered [ 5347.995579] LNet: 161405:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5347.999046] LNetError: 161405:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5348.021622] LNet: Removed LNI 192.168.202.143@tcp [ 5348.793912] Key type .llcrypt unregistered [ 5348.795928] Key type ._llcrypt unregistered [ 5372.601497] Key type ._llcrypt registered [ 5372.602609] Key type .llcrypt registered [ 5372.763283] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5373.837481] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5374.149175] alg: No test for adler32 (adler32-zlib) [ 5375.388886] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 5375.664880] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5377.376181] Key type lgssc registered [ 5378.428418] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5385.482629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5385.505122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5386.943879] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5391.579548] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5397.476581] LustreError: 162804:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5397.504810] LustreError: 162804:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 5399.139345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5399.400750] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5403.410066] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5408.788531] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5412.636612] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5418.925519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5419.272575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5424.750409] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5430.299165] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5433.661754] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5443.205970] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5443.244370] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5443.261709] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5443.273918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.292924] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5444.269701] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5444.287301] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5444.298445] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5444.306097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.314966] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5445.350914] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5445.370221] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5445.378715] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5445.382277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.388907] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5446.349197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.545153] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5447.567436] Lustre: Skipped 1 previous similar message [ 5447.594616] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5447.609953] Lustre: Skipped 1 previous similar message [ 5447.614761] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5447.622183] Lustre: Skipped 1 previous similar message [ 5447.629277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.644311] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5447.651660] Lustre: Skipped 1 previous similar message [ 5448.657362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.476394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.354757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.304263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.160657] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5452.166597] Lustre: Skipped 4 previous similar messages [ 5452.184866] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5452.195260] Lustre: Skipped 4 previous similar messages [ 5452.198932] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5452.205629] Lustre: Skipped 4 previous similar messages [ 5452.212548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.223246] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5452.229479] Lustre: Skipped 4 previous similar messages [ 5453.535061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.505120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.402982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.205421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.135085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.159525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.131656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.978402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.112963] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5461.117638] Lustre: Skipped 8 previous similar messages [ 5461.136497] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5461.143786] Lustre: Skipped 8 previous similar messages [ 5461.147190] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5461.150088] Lustre: Skipped 8 previous similar messages [ 5461.152849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.158251] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5461.161369] Lustre: Skipped 8 previous similar messages [ 5461.929798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.955169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.808716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.752665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.925418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.955353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.925801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.117424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.225348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.183120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.193732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.270183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.149434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.173911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.935944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.091104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.145256] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5478.150350] Lustre: Skipped 16 previous similar messages [ 5478.180320] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5478.188214] Lustre: Skipped 16 previous similar messages [ 5478.192242] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5478.197347] Lustre: Skipped 16 previous similar messages [ 5478.201323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.208954] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5478.214776] Lustre: Skipped 16 previous similar messages [ 5479.037782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.108561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.131077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.378885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.405364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.512954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.471416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.470589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.599571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.595556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.440824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.361705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.300499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.553867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.550935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.559963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.536403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.543925] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5496.564976] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5496.579855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.404510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.308435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.117740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.946847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.789382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.720557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.723416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.727878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.649930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.720627] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5505.738838] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5505.744104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.728383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.768063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.798563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.800581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.634774] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5510.640081] Lustre: Skipped 32 previous similar messages [ 5510.654655] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5510.662201] Lustre: Skipped 32 previous similar messages [ 5510.670299] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5510.674742] Lustre: Skipped 32 previous similar messages [ 5510.678925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.688743] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5510.690960] Lustre: Skipped 32 previous similar messages [ 5511.625777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.377849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.347214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.259642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.327639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.286193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.230375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.017278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.956185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.700376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.661928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.564866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.478819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.190194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.024157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.846250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.727217] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5525.735512] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5525.737834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.564940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.491892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.252346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.951801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.702787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.677679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.414275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.335922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.117254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.104955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.982467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.901981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.654497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.529234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.472107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.429270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.371851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.438219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.546385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.471145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.170614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.086567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.950368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.735467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.702603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.470264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.440973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.141584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.005366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.034366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.947379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.838904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.784339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.500828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.363247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.117686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.041125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.918991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.854832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.799161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.623286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.356501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.140194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.899378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.660167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.629508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.573177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.503678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.385844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.400712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.245638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.199929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.020154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.896932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.894674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.858334] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5574.863909] Lustre: Skipped 72 previous similar messages [ 5574.876902] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5574.885668] Lustre: Skipped 72 previous similar messages [ 5574.889744] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5574.895536] Lustre: Skipped 72 previous similar messages [ 5574.900245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.909321] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5574.915711] Lustre: Skipped 72 previous similar messages [ 5575.753828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.559195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.349899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.101954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.930762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.848895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.707694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.542336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.217772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.921705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.743978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.475165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.162598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.818612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.615478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.506869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.225467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.907006] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.684360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.549713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.367807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.164098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.028924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.818284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.605980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.319398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.086533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.907796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.769100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.528539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.403884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.188206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.983944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.782547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.631678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.366409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.126486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.914075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.748761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.372623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.164689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.928796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.817053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.756536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.529307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.423395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.149580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.916241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.655515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.377960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.109053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.824139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.562919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.196984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.823405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.479380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.244701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.089876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.830433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.628892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.777477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.639746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.563545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.591170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.650789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.570177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.516752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.384724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.280213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.189676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.371753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.382813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.454640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.534831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.464921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.583616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.486983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.711853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.897300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.843685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.841140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.811522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.711896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.648542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.565360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.445366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.295272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.221342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.033653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.865094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.657389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.514106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.552839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.566846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.569072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.401580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.480352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.393292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.317217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.150562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.087931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.866910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.706791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.629606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.595306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.381742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.189632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.908808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.807677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.589646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.615243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.459549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.347231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.246493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.295858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.163646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.029901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.846341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.882934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.778828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.575785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.445622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.300527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.099745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.172674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.941859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.892695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.695232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.804891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.897132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.885720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.946162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.778855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.835080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.596897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.668190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.487160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.455429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.373570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.190554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.017640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.940845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.816751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.513630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.285819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.081061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.903454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.502634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.214739] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5703.218258] Lustre: Skipped 148 previous similar messages [ 5703.239629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5703.248376] Lustre: Skipped 148 previous similar messages [ 5703.261324] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5703.275228] Lustre: Skipped 148 previous similar messages [ 5703.286709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.300715] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5703.307090] Lustre: Skipped 148 previous similar messages [ 5704.014867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.741465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.419715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.283420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.079590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.830175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.586531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.218477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.942580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.575960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.279711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.026990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.758597] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5712.768317] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5712.769063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.547041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.309857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.010205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.573592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.373402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.042465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.884907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.678934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.523723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.364448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.217922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.925543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.750799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.488180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.342544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.040407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.840856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.625377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.476564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.202540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.025878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.860721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.643512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.625587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.497192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.357083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.242229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.044363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.856573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.558247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.266319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.073991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.867965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.626640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.413776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.316128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.099209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.842113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.502767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.344890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.160397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.919647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.761628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.602840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.461336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.312797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.080465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.807592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.617570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.215925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.034755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.721386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.433408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.985187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.869395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.574599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.362541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.988763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.823892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.448528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.160525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.852366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.577251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.285446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.977254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.568357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.194563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.891505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.672103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.355793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.986420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.645769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.393499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.150696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.864860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.475197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.320101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.072458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.711967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.361267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.232761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.943725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.695487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.390113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.034024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.710375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.449933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.165449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.834218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.456548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.079894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.748905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.530101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.188809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.111649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.927925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.800486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.560920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.279750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.182619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.997307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.891275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.709499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.640807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.512558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.415957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.207989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.073965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.821330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.607853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.383512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.233244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.994512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.735102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.505760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.437651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.291749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.056208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.837327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.609875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.339403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.178455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.971485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.700529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.396219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.065739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.776924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.468198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.337669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.055753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.831434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.624542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.462137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.119513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.946210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.743668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.641398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.451172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.283433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.047619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.913416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.748758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.571299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.245294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.982206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.767539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.571575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.298235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.055856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.806582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.449326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.157669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.834380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.562337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.233488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.967587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.783250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.609176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.297106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.102066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.880801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.481415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.205329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.988400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.703110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.464122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.177348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.910802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.567466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.265950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.805215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.456226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.106576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.939139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.608427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.300797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.025445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.799891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.465194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.225259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.972181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.695247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.471606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.163615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.074389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.789930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.577499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.290394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.051883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.908635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.720929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.509541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.192234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.999354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.773408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.627927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.521459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.183216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.931580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.648970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.391947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.078948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.952245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.668789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.430596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.101873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.926064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.586484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.373795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.060689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.840368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.583053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.337159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.993965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.824429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.602940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.441109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.131141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.945172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.604869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.358171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.162341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.025709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.942208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.608641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.475708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.216827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.951347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.760308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.600565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.346829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.112750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.710530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.410315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.113641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.819852] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5891.827959] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5891.830874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.661198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.473661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.064876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.782765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.433738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.138105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.848815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.597634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.292773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.055091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.987464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.873607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.488743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.195285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.845129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.532395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.310393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.040047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.874208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.545220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.373619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.122472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.896514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.687166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.448498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.211121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.938691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.671831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.419349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.076798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.718648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.397339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.123633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.906627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.688223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.555959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.331543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.089381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.809741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.539074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.455427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.128358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.809189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.402382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.201487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.810265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.626080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.354450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.150885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.832573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.593758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.238600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.071911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.667573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.382422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.124952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.904480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.579286] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5934.587789] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5934.591193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.260638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.013380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.889292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.687546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.305304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.981923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.752539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.582246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.323645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.155343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.933120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.651940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.478204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.147222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.786842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.491536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.204989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.005518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.792946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.566323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.205819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.813154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.483442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.205554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.898272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.632846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.278720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.954401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.665600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.359658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.054477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.869242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.601420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.244876] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5959.247731] Lustre: Skipped 340 previous similar messages [ 5959.256522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5959.262550] Lustre: Skipped 340 previous similar messages [ 5959.265571] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5959.268721] Lustre: Skipped 340 previous similar messages [ 5959.271478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.068295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.075555] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5960.079847] Lustre: Skipped 341 previous similar messages [ 5960.835524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.645681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.320534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.019876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.698541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.297953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.890536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.618199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.341086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.940206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.564548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.209256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.782273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.326061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.899053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.449981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.146161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.869741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.554421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.240327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.148837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.854883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.519532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.109653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.740719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.387220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.176523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.826649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.584454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.169760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.873119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.545071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.256102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.878188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.519243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.165102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.885084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.521063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.202476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.890806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.582810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.130933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.818130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.479501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.106794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.715237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.382824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.092770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.811891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.412825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.119773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.762330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.526092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.370396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.314433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.072317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.799253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.540593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.240635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.806555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.423512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.111887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.782670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.311218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.821672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.355639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.993963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.738431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.345749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.967888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.600869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.293467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.000571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.688074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.337693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.019611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.658606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.380438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.038118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.796476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.523459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.323589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.952160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.546848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.194614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.867284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.540495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.297266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.006737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.634740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.322833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.946848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.552945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.256591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.856086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.519930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.291041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.974787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.643153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.315699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.006355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.669344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.248770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.903275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.696624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.539858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.270571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.068881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.811548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.498875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.158839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.868754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.742256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.540622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.354582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.999110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.777436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.487733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.208158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.883611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.522208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.167638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.892068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.499878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.143160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.757882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.320724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.914603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.595546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.182864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.854072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.415867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.083841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.629474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.306448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.950309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.665898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.378347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.125872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.795639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.520773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.249647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.971240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.639545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.334674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.894654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.457821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.920476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.550184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.167249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.737239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.277363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.833324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.371271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.980851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.609336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.313135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.999433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.709444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.482329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.139892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.787421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.455568] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6069.464111] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6069.473372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.128116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.723246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.438472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.125914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.706477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.242517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.809365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.390823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.917642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.530938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.235041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.853873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.511200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.230490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.842670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.521466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.073930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.650365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.301762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.946646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.509755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.245634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.956305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.645461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.286936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.818508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.401188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.933800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.513145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.124810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.761223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.384830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.923582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.497313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.215082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.887017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.509350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.052751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.562655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.110857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.671806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.250873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.996871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.762526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.303272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.921121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.644491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.319358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.018754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.717549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.443546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.139581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.750604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.455673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.175638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.841721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.535348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.347888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.960547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.566868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.251490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.948588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.655951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.351863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.977340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.582204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.185823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.842373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.449935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.123306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.742763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.468577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.101685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.686773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.342864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.046441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.870855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.609885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.265640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.960482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.537765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.106444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.709799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.274910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.912903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.450822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.084513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.685241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.225554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.788976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.352675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.885743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.519171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.112493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.728143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.245658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.790910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.367084] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6131.372806] LustreError: 162823:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6131.373095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.024087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.647165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.208173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.811445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.502190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.069329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.652835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.217469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.769810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.356768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.922301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.420655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.912417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.467664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.012962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.568299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.122911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.707560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.325073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.946237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.465754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.015894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.594310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.161337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.721545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.298604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.927992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.469947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.012141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.501160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.995643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.489863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.028314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.549530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.108475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.599564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.084626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.564750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.032318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.547472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.091619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.636370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.254388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.776788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.335149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.914777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.522706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.040866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.609666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.157439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.811261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.428437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.063191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.654386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.251216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.820252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.406561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.942062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.503927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.135436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.792191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.414317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.972026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.609498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.283340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.924566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.596596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.176798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.795131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.386043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.998167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.569954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.176393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.753758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.403853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.030144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.670061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.207523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.735744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.313206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.913526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.529458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.291236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.006831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.830795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.493833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.343141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.056225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.821477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.509325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.365137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.110531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.769060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.492487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.085413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.725563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.330993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.955045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.538220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.225234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.952413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.609520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.227238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.980810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.633259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.260979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.820549] LustreError: 164952:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6532.824857] LustreError: 162823:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6916.581459] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6916.592049] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6916.618073] Lustre: Skipped 368 previous similar messages [ 6916.624427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6918.632836] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6921.564696] Lustre: server umount lustre-OST0000 complete [ 6928.071921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6928.085530] Lustre: Skipped 1 previous similar message [ 6933.473960] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6933.479104] Lustre: Skipped 2 previous similar messages [ 6934.993126] Lustre: server umount lustre-MDT0000 complete [ 6938.596191] LustreError: 162804:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6938.615892] LustreError: 162804:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 6940.031931] LustreError: 162783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777001149 with bad export cookie 17026819336093605598 [ 6940.036130] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6940.043913] LustreError: 162783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6940.611452] Lustre: server umount lustre-MDT0001 complete [ 6951.849741] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 6954.266745] Key type lgssc unregistered [ 6954.573990] LNet: 217646:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6954.581304] LNetError: 217646:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6954.605586] LNet: Removed LNI 192.168.202.143@tcp [ 6955.798413] Key type .llcrypt unregistered [ 6955.804417] Key type ._llcrypt unregistered [ 6982.795316] Key type ._llcrypt registered [ 6982.799643] Key type .llcrypt registered [ 6983.033546] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 6984.850113] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6984.957841] alg: No test for adler32 (adler32-zlib) [ 6986.168227] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 6986.738288] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 6988.520363] Key type lgssc registered [ 6990.094052] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7000.185787] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7000.236085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7002.080645] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7007.142972] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7012.324707] LustreError: 219050:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7012.354209] LustreError: 219050:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 7017.448151] LustreError: 219049:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7018.184624] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7018.571653] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7023.896340] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7031.109424] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7034.914113] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7042.179083] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7042.571353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7047.796947] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 7050.187846] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7057.163280] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7063.122349] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7064.928252] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7064.947280] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7064.959539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7064.989108] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7064.993802] LustreError: 219069:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7068.655308] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7068.662946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7068.675484] Lustre: Skipped 1 previous similar message [ 7068.683213] Lustre: Skipped 1 previous similar message [ 7070.168131] Lustre: lustre-OST0000: Not available for connect from 192.168.202.43@tcp (stopping) [ 7073.693693] Lustre: server umount lustre-OST0000 complete [ 7085.056112] LustreError: 219069:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7109.140332] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7109.520896] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7116.734819] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7123.353164] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7125.476782] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7130.303811] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9061a091f000.ost_server_uuid 50 [ 7132.050201] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9061a091f000.ost_server_uuid in FULL state after 0 sec [ 7133.045459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7133.059086] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7133.060060] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7133.080919] Lustre: Skipped 1 previous similar message [ 7138.275093] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7138.288793] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7138.294516] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7138.303207] Lustre: Skipped 1 previous similar message [ 7138.318612] Lustre: Skipped 1 previous similar message [ 7138.987781] Lustre: server umount lustre-OST0000 complete [ 7143.397612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7143.399473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7143.409497] Lustre: Skipped 1 previous similar message [ 7143.430872] Lustre: Skipped 1 previous similar message [ 7148.515237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7148.527311] Lustre: Skipped 1 previous similar message [ 7149.872605] Lustre: server umount lustre-MDT0000 complete [ 7153.635204] LustreError: 219050:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7153.659498] LustreError: 219050:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 7154.365728] LustreError: 221441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777001363 with bad export cookie 2067464864776864551 [ 7154.368462] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7154.378862] LustreError: 221441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7154.750762] Lustre: server umount lustre-MDT0001 complete [ 7166.581932] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 7169.918118] Key type lgssc unregistered [ 7170.421607] LNet: 223696:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7170.432248] LNetError: 223696:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7170.458043] LNet: Removed LNI 192.168.202.143@tcp [ 7171.472084] Key type .llcrypt unregistered [ 7171.473635] Key type ._llcrypt unregistered [ 7200.780171] Key type ._llcrypt registered [ 7200.782937] Key type .llcrypt registered [ 7200.875699] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 7201.726059] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7201.759090] alg: No test for adler32 (adler32-zlib) [ 7202.944174] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 7203.159381] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 7204.808191] Key type lgssc registered [ 7205.946720] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7215.318127] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7215.330947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7216.827523] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7221.238848] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7226.374784] LustreError: 225388:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7226.398537] LustreError: 225388:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 7230.025646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7230.547050] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7235.040822] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7238.170786] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7245.799179] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7246.491492] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7253.573354] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7260.870538] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 7262.681983] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7262.857520] Lustre: lustre-OST0001: new disk, initializing [ 7262.861596] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7262.979432] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7269.326777] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7271.106389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7271.111692] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7271.209334] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7279.562951] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7285.020735] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7307.242889] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7307.250756] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7307.254378] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7307.286884] Lustre: Skipped 3 previous similar messages [ 7310.130145] Lustre: server umount lustre-MDT0000 complete [ 7312.362928] LustreError: 225387:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7312.387104] LustreError: 225387:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 7313.684518] LustreError: 225368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777001522 with bad export cookie 8795877411304460735 [ 7313.692614] LustreError: 225368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7313.699102] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7314.093125] Lustre: server umount lustre-MDT0001 complete [ 7328.472862] Lustre: server umount lustre-OST0000 complete [ 7341.064751] Lustre: server umount lustre-OST0001 complete [ 7360.211745] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 7369.537322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7369.857912] LustreError: 230717:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7369.946769] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7373.739523] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7375.340934] LustreError: 230718:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7380.452789] LustreError: 230717:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7381.778177] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7382.096271] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7385.597172] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7388.411795] Lustre: 231827:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7394.593443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7395.883890] LustreError: 232181:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7395.900275] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 7401.364398] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7409.485477] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7409.751302] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7409.763551] Lustre: Skipped 1 previous similar message [ 7414.852474] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7420.998489] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7424.205838] Lustre: 233661:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7443.433966] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7443.436698] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7443.438435] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7443.438442] Lustre: Skipped 3 previous similar messages [ 7443.444974] Lustre: Skipped 1 previous similar message [ 7445.133392] Lustre: server umount mylustre-MDT0000 complete [ 7448.249230] LustreError: 230693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777001657 with bad export cookie 8795877411304463542 [ 7448.252748] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7448.258232] LustreError: 230693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7448.545708] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7448.548156] LustreError: 230712:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7448.558350] Lustre: Skipped 3 previous similar messages [ 7448.573769] LustreError: 230712:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 5 previous similar messages [ 7448.626548] Lustre: server umount mylustre-MDT0001 complete [ 7457.981419] Lustre: server umount mylustre-OST0000 complete [ 7461.686570] Lustre: server umount mylustre-OST0001 complete [ 7483.248156] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 7494.186983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7494.506176] LustreError: 235951:0:(ldlm_lib.c:1180:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7494.615481] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7498.742317] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7507.494081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7511.879343] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7514.815383] Lustre: 237058:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7522.086834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7528.697148] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7538.411948] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7539.712825] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7539.717018] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49793) [ 7545.847060] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7553.761047] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7557.215710] Lustre: 238896:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7570.408350] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7570.420688] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7570.429960] Lustre: Skipped 3 previous similar messages [ 7570.449765] Lustre: Skipped 6 previous similar messages [ 7571.891344] Lustre: server umount tfs-MDT0000 complete [ 7575.525104] LustreError: 236667:0:(ldlm_lib.c:1180:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7575.556949] LustreError: 236667:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 6 previous similar messages [ 7576.642788] LustreError: 235928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777001785 with bad export cookie 8795877411304465208 [ 7576.651436] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7576.655106] LustreError: 235928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7577.034815] Lustre: server umount tfs-MDT0001 complete [ 7590.979220] Lustre: server umount tfs-OST0000 complete [ 7605.447648] Lustre: server umount tfs-OST0001 complete [ 7631.404464] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 7642.875727] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7643.776444] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7643.791085] Lustre: Skipped 3 previous similar messages [ 7648.831644] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7658.074045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7663.057535] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7665.784372] Lustre: 242242:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7672.038317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7678.615849] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7687.356756] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7688.591290] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7688.620680] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49825) [ 7694.204963] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7702.229552] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7705.551765] Lustre: 244078:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7717.854919] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:38:44 (1777001924) [ 7734.759913] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7734.762763] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7734.778273] Lustre: Skipped 2 previous similar messages [ 7734.797505] Lustre: Skipped 3 previous similar messages [ 7739.839758] Lustre: server umount lustre-MDT0000 complete [ 7739.881123] LustreError: 241135:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7739.900942] LustreError: 241135:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 8 previous similar messages [ 7743.787559] LustreError: 241109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777001952 with bad export cookie 8795877411304466874 [ 7743.791496] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7743.797626] LustreError: 241109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7744.993492] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7745.008827] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7745.017714] Lustre: Skipped 2 previous similar messages [ 7745.037575] Lustre: Skipped 1 previous similar message [ 7754.152824] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7754.158415] Lustre: Skipped 3 previous similar messages [ 7758.304300] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7758.665789] Lustre: server umount lustre-MDT0001 complete [ 7768.285868] Lustre: server umount lustre-OST0000 complete [ 7778.324499] Lustre: server umount lustre-OST0001 complete [ 7784.279040] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 7793.068750] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 7831.979390] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7831.979390] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7831.979390] [ 7832.020268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7832.253472] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7832.279184] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7832.361972] Lustre: lustre-MDT0000: new disk, initializing [ 7832.460876] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7832.463115] Lustre: Skipped 3 previous similar messages [ 7832.487800] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7836.605354] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7847.001085] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7847.001085] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7847.001085] [ 7847.032226] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7847.163786] Lustre: 247231:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7847.261943] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7847.270893] Lustre: Skipped 1 previous similar message [ 7847.404512] Lustre: lustre-MDT0001: new disk, initializing [ 7847.557217] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7847.568981] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7851.777725] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7861.870896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7862.125899] Lustre: lustre-OST0000: new disk, initializing [ 7862.129535] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7863.365568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7863.379576] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7863.509410] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7868.466396] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7878.818402] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7878.990308] Lustre: lustre-OST0001: new disk, initializing [ 7878.996067] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7880.224619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7880.229610] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7880.396104] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7885.370728] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 7895.912388] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 23:41:42 (1777002102) [ 7907.220107] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 23:41:54 (1777002114) [ 7925.217361] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7925.235633] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7925.237721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7925.247704] Lustre: Skipped 1 previous similar message [ 7925.266516] Lustre: Skipped 1 previous similar message [ 7927.686473] Lustre: server umount lustre-OST0000 complete [ 7934.436518] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7934.447272] LustreError: Skipped 1 previous similar message [ 7937.639495] Lustre: server umount lustre-MDT0000 complete [ 7942.070482] LustreError: 247221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777002151 with bad export cookie 8795877411304468428 [ 7942.075784] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7942.079568] LustreError: 247221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7948.473105] Lustre: server umount lustre-MDT0001 complete [ 7958.715366] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 7975.334098] Lustre: server umount lustre-OST0001 complete [ 7981.553723] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 7994.086380] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 8028.131920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8028.402685] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8028.430530] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8028.507610] Lustre: lustre-MDT0000: new disk, initializing [ 8028.632500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8032.725564] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8043.082177] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8043.200847] Lustre: 254097:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8043.517954] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8047.436764] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8054.098380] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8057.816914] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8064.416549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8064.718546] Lustre: lustre-OST0000: new disk, initializing [ 8064.723773] Lustre: Skipped 1 previous similar message [ 8064.734769] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8064.743933] Lustre: Skipped 2 previous similar messages [ 8066.694387] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8066.706437] Lustre: Skipped 1 previous similar message [ 8066.723367] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8066.835622] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8071.178784] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8078.181719] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8087.541074] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8087.548638] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8087.557429] LustreError: Skipped 1 previous similar message [ 8087.562184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8087.562198] Lustre: Skipped 6 previous similar messages [ 8087.602366] Lustre: Skipped 5 previous similar messages [ 8090.479997] Lustre: server umount lustre-OST0000 complete [ 8099.925223] Lustre: server umount lustre-MDT0000 complete [ 8102.887361] LustreError: 256395:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8102.903509] LustreError: 256395:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 16 previous similar messages [ 8103.041236] LustreError: 254088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777002312 with bad export cookie 8795877411304469898 [ 8103.044474] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8103.048379] LustreError: 254088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8103.339239] Lustre: server umount lustre-MDT0001 complete [ 8111.842596] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 8114.114681] Key type lgssc unregistered [ 8114.340847] LNet: 257468:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8114.346521] LNetError: 257468:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8115.372900] LNet: Removed LNI 192.168.202.143@tcp [ 8116.085205] Key type .llcrypt unregistered [ 8116.089498] Key type ._llcrypt unregistered [ 8137.678961] Key type ._llcrypt registered [ 8137.680890] Key type .llcrypt registered [ 8137.766592] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 8138.568024] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8138.589302] alg: No test for adler32 (adler32-zlib) [ 8139.673280] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 8139.864469] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 8141.560370] Key type lgssc registered [ 8142.518480] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8149.845572] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8149.880060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8151.293756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8155.660059] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8161.768617] LustreError: 258894:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8161.780052] LustreError: 258894:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 8163.854280] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8164.163829] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8168.019688] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8172.855730] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8175.909705] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8181.814837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8182.200541] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8188.389934] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8193.974878] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8194.886517] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8196.546894] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8199.652258] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8199.663838] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8199.679114] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8201.698496] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8201.708540] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8201.708792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8201.730296] Lustre: Skipped 1 previous similar message [ 8203.165387] Lustre: server umount lustre-OST0000 complete [ 8206.819679] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8206.827311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8206.845505] Lustre: Skipped 1 previous similar message [ 8206.863469] Lustre: Skipped 1 previous similar message [ 8211.937613] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8211.943630] Lustre: Skipped 1 previous similar message [ 8212.843568] Lustre: server umount lustre-MDT0000 complete [ 8216.062265] LustreError: 258873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777002425 with bad export cookie 13547898498492921302 [ 8216.066571] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8216.068826] LustreError: 258873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8216.330979] Lustre: server umount lustre-MDT0001 complete [ 8225.224634] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 8227.669596] Key type lgssc unregistered [ 8227.991394] LNet: 262123:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8227.996431] LNetError: 262123:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8228.014220] LNet: Removed LNI 192.168.202.143@tcp [ 8228.819861] Key type .llcrypt unregistered [ 8228.821818] Key type ._llcrypt unregistered [ 8242.166757] Key type ._llcrypt registered [ 8242.169533] Key type .llcrypt registered [ 8242.262681] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 8243.071137] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8243.197553] alg: No test for adler32 (adler32-zlib) [ 8244.199440] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 8244.444564] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 8246.088175] Key type lgssc registered [ 8247.092775] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8254.378636] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8254.397344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8269.920534] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8275.040274] LustreError: 263279:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 8301.793164] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8306.664631] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8311.240851] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8313.030767] Lustre: server umount lustre-OST0000 complete [ 8325.590436] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 8328.015723] Key type lgssc unregistered [ 8328.362239] LNet: 264781:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8328.369085] LNetError: 264781:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8328.384553] LNet: Removed LNI 192.168.202.143@tcp [ 8329.190772] Key type .llcrypt unregistered [ 8329.193439] Key type ._llcrypt unregistered [ 8359.341967] Key type ._llcrypt registered [ 8359.343879] Key type .llcrypt registered [ 8359.417377] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 8360.261469] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8360.307316] alg: No test for adler32 (adler32-zlib) [ 8361.517412] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 8361.710670] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 8363.360213] Key type lgssc registered [ 8364.357767] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8375.805333] loop0: detected capacity change from 0 to 204800000 [ 8376.032059] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8376.042049] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8376.305697] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8377.261653] loop0: detected capacity change from 0 to 204800000 [ 8377.438693] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8377.451805] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8377.816436] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8378.784108] loop0: detected capacity change from 0 to 204800000 [ 8378.941832] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8378.949840] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8379.033281] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8379.942980] loop0: detected capacity change from 0 to 204800000 [ 8380.079916] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8380.984294] loop0: detected capacity change from 0 to 204800000 [ 8381.030541] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8384.264501] loop0: detected capacity change from 0 to 204800000 [ 8384.291207] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8387.482041] loop0: detected capacity change from 0 to 204800000 [ 8387.519381] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8390.756333] loop0: detected capacity change from 0 to 204800000 [ 8390.774831] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8393.700595] loop0: detected capacity change from 0 to 204800000 [ 8393.767985] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8393.825702] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8393.891580] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8393.901716] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8393.906696] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8393.911265] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8393.918607] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8393.929748] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8393.935472] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8393.939208] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8393.943199] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8393.947026] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8393.951388] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8393.955704] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8393.960228] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8393.967066] Lustre: 268156:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8398.777938] Lustre: server umount MGS complete [ 8399.512201] loop0: detected capacity change from 0 to 204800000 [ 8399.593163] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8399.921053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8399.965517] Lustre: lustre-MDT0000: Aborting client recovery [ 8400.760121] loop1: detected capacity change from 0 to 204800000 [ 8400.859236] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8400.994576] Lustre: 268588:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8401.005901] Lustre: 268588:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8401.014238] Lustre: 268588:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8401.021600] Lustre: 268588:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8401.031887] Lustre: 268588:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8401.044483] Lustre: 268588:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8401.050136] Lustre: 268588:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8401.066221] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8401.258291] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8401.304608] Lustre: lustre-MDT0001: Aborting client recovery [ 8402.094968] loop2: detected capacity change from 0 to 204800000 [ 8402.224193] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8402.337666] Lustre: 268666:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8402.351658] Lustre: 268666:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8402.368444] Lustre: 268666:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8402.379907] Lustre: 268666:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8402.392585] Lustre: 268666:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8402.400139] Lustre: 268666:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8402.408596] Lustre: 268666:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8402.499393] Lustre: Found index 0 for lustre-OST0000, updating log [ 8402.584984] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8402.595277] Lustre: lustre-OST0000: Aborting recovery [ 8403.398279] loop3: detected capacity change from 0 to 204800000 [ 8403.559085] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8403.602684] Lustre: 268750:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8403.610293] Lustre: 268750:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8403.619597] Lustre: 268750:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8403.627261] Lustre: 268750:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8403.634847] Lustre: 268750:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8403.642241] Lustre: 268750:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8403.647548] Lustre: 268750:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8403.656951] Lustre: Found index 1 for lustre-OST0001, updating log [ 8403.759468] Lustre: lustre-OST0001: Aborting recovery [ 8409.087595] LustreError: 268532:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8410.095723] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8410.115445] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8410.119963] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8410.137192] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8411.114414] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8411.115305] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8416.160528] LustreError: 268499:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 153us: rc = -2 [ 8416.691383] LustreError: 268500:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 452us: rc = -2 [ 8416.705873] LustreError: 268500:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 35 previous similar messages [ 8431.585944] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8431.586539] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8431.591953] Lustre: Skipped 2 previous similar messages [ 8431.594350] Lustre: Skipped 1 previous similar message [ 8434.463243] Lustre: server umount lustre-MDT0000 complete [ 8435.085945] LustreError: 268481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777002644 with bad export cookie 14261568748996592821 [ 8435.093460] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8436.706637] LustreError: 268498:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8436.708022] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8436.712474] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8436.729864] LustreError: 268498:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 8436.737110] Lustre: Skipped 2 previous similar messages [ 8440.736657] LustreError: 269041:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8440.739852] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8440.750469] LustreError: 269041:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 8440.753102] Lustre: Skipped 2 previous similar messages [ 8441.441870] Lustre: server umount lustre-MDT0001 complete [ 8448.171729] Lustre: server umount lustre-OST0000 complete [ 8463.328124] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8463.461996] Lustre: server umount lustre-OST0001 complete [ 8467.777936] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:51:15 (1777002675) [ 8483.979366] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 8489.031798] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 8510.121932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8510.250812] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8510.274918] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8510.315383] Lustre: lustre-MDT0000: new disk, initializing [ 8510.359414] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8510.363213] Lustre: Skipped 1 previous similar message [ 8510.371210] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8512.911328] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8520.587110] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8520.656887] Lustre: 272782:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8520.739205] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8520.741076] Lustre: Skipped 1 previous similar message [ 8520.813910] Lustre: lustre-MDT0001: new disk, initializing [ 8520.891209] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8520.923886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8520.933861] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8523.767816] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8529.002358] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8531.619290] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8536.169697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8536.325675] Lustre: lustre-OST0000: new disk, initializing [ 8536.328736] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8536.386445] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8540.368917] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8544.302286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8544.322544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8544.375747] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8545.687299] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8549.237653] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8558.222565] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8563.333875] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8568.389917] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8605.669349] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8605.673050] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8605.676654] Lustre: Skipped 1 previous similar message [ 8609.319977] Lustre: server umount lustre-OST0000 complete [ 8613.345332] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8613.355175] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8613.367903] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8613.371175] Lustre: Skipped 1 previous similar message [ 8625.124189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8625.129493] Lustre: Skipped 5 previous similar messages [ 8626.144129] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8626.254349] Lustre: server umount lustre-MDT0000 complete [ 8628.541249] LustreError: 273560:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777002837 with bad export cookie 14261568748996595530 [ 8628.547318] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8628.549709] LustreError: 273560:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8628.779686] Lustre: server umount lustre-MDT0001 complete [ 8634.014553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8636.767362] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8639.668294] Lustre: server umount MGS complete [ 8646.787782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8647.139334] LustreError: 278253:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8647.152286] LustreError: 278266:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8647.197485] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8649.679638] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8652.265231] LustreError: 278254:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8655.680526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8658.787571] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8662.315973] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8664.300596] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8668.687771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8672.717687] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8676.406407] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8702.955126] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8702.963081] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8702.970173] Lustre: Skipped 1 previous similar message [ 8702.975873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8704.561761] Lustre: server umount lustre-OST0000 complete [ 8707.046486] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8721.376197] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8721.462463] Lustre: server umount lustre-MDT0000 complete [ 8722.402745] LustreError: 278253:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8722.416602] LustreError: 278253:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 8723.424140] Lustre: 266119:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1777002916/real 1777002916] req@0000000053d0f13f x1863322247013248/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1777002932 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 [ 8723.424221] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8723.436880] Lustre: 266119:0:(client.c:2479:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 8723.445700] Lustre: Skipped 2 previous similar messages [ 8723.681154] LustreError: 278236:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777002932 with bad export cookie 14261568748996597350 [ 8723.684293] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8723.688032] LustreError: 278236:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8723.906692] Lustre: server umount lustre-MDT0001 complete [ 8730.572197] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 8732.212834] Key type lgssc unregistered [ 8732.439814] LNet: 281837:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8732.445863] LNetError: 281837:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8732.456987] LNet: Removed LNI 192.168.202.143@tcp [ 8732.989539] Key type .llcrypt unregistered [ 8732.991070] Key type ._llcrypt unregistered [ 8755.441773] Key type ._llcrypt registered [ 8755.444029] Key type .llcrypt registered [ 8755.504093] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 8764.381245] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 8764.973384] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8765.033038] alg: No test for adler32 (adler32-zlib) [ 8765.994463] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 8766.155831] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 8767.760183] Key type lgssc registered [ 8768.469934] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8789.439596] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8789.465050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8790.639755] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8790.666103] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8790.724373] Lustre: lustre-MDT0000: new disk, initializing [ 8790.798153] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8790.813349] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8793.194400] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8800.300244] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8800.390881] Lustre: 285734:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8800.444976] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8800.448294] Lustre: Skipped 1 previous similar message [ 8800.519801] Lustre: lustre-MDT0001: new disk, initializing [ 8800.591799] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8800.613182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8800.619386] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8803.288600] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8808.044546] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8810.245857] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8814.773084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8814.955681] Lustre: lustre-OST0000: new disk, initializing [ 8814.960158] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8815.024514] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8818.844526] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8820.756429] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8820.762786] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8820.826775] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8823.640794] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8828.184441] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8831.637555] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8837.095096] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8887.266337] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8887.270160] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8887.270729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8887.278177] Lustre: Skipped 1 previous similar message [ 8890.182589] Lustre: server umount lustre-OST0000 complete [ 8892.897257] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8892.900414] LustreError: Skipped 1 previous similar message [ 8892.903038] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8892.912273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8892.915384] Lustre: Skipped 1 previous similar message [ 8897.505391] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8897.507526] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8898.724424] Lustre: server umount lustre-MDT0000 complete [ 8900.776069] LustreError: 285727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777003109 with bad export cookie 8578574039846780601 [ 8900.776806] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8900.783759] LustreError: 285727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8901.008167] Lustre: server umount lustre-MDT0001 complete [ 8905.854573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8907.820917] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8910.683494] Lustre: server umount MGS complete [ 8916.461324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8916.676411] LustreError: 291124:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8916.705773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8918.736102] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8922.082505] LustreError: 291111:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8922.090748] LustreError: 291111:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 8923.520679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8923.696577] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8926.088635] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8929.694331] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8931.771300] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8935.705613] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8935.926700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8939.451220] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 8943.340970] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8961.504960] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8961.512416] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8961.523173] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8961.525940] Lustre: Skipped 1 previous similar message [ 8963.562227] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8967.596118] Lustre: server umount lustre-OST0000 complete [ 8973.795226] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8973.795956] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8973.807853] Lustre: Skipped 2 previous similar messages [ 8973.811239] Lustre: Skipped 2 previous similar messages [ 8976.136084] Lustre: server umount lustre-MDT0000 complete [ 8978.336495] LustreError: 291092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777003187 with bad export cookie 8578574039846782435 [ 8978.337461] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8978.342556] LustreError: 291092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8978.570892] Lustre: server umount lustre-MDT0001 complete [ 8984.354273] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 8986.004357] Key type lgssc unregistered [ 8986.177478] LNet: 294698:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8986.181833] LNetError: 294698:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8986.197849] LNet: Removed LNI 192.168.202.143@tcp [ 8986.642557] Key type .llcrypt unregistered [ 8986.643789] Key type ._llcrypt unregistered [ 9000.229232] Key type ._llcrypt registered [ 9000.230601] Key type .llcrypt registered [ 9000.295367] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 9009.402615] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 9010.109886] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9010.201868] alg: No test for adler32 (adler32-zlib) [ 9011.147824] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 9011.299225] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 9012.928169] Key type lgssc registered [ 9013.571459] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9027.431384] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 9035.535606] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9035.557423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9036.727700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9036.745260] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9036.806547] Lustre: lustre-MDT0000: new disk, initializing [ 9036.871457] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9036.881238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9039.407409] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9045.860216] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9046.029327] Lustre: lustre-OST0000: new disk, initializing [ 9046.032987] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9046.036270] Lustre: Skipped 1 previous similar message [ 9046.090061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9049.091921] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9049.138182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9049.145973] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9049.163648] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9472.761921] Lustre: server umount lustre-MDT0000 complete [ 9474.704659] LustreError: 297863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777003683 with bad export cookie 14719561032565201822 [ 9474.710963] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9484.338326] Lustre: server umount lustre-OST0000 complete [ 9496.086574] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 9500.191053] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 9517.519135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9517.654381] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9517.671927] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9517.716461] Lustre: lustre-MDT0000: new disk, initializing [ 9517.744424] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9517.752156] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9519.655699] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9525.388195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9525.454730] Lustre: 302882:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9525.493388] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9525.496277] Lustre: Skipped 1 previous similar message [ 9525.535196] Lustre: lustre-MDT0001: new disk, initializing [ 9525.596110] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9525.614563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9525.621919] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9527.569133] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9531.748090] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9533.565471] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9536.722109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9536.861876] Lustre: lustre-OST0000: new disk, initializing [ 9536.864450] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9536.923444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9537.184055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9537.191718] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9537.252151] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9539.539955] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9543.564434] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9545.478241] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9545.594954] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9547.373176] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9547.462112] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9552.866019] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9552.868762] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9552.871810] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9552.877931] Lustre: Skipped 1 previous similar message [ 9552.881778] Lustre: Skipped 1 previous similar message [ 9554.407914] Lustre: server umount lustre-OST0000 complete [ 9556.449251] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9556.452914] LustreError: Skipped 1 previous similar message [ 9556.454588] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9556.461481] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9557.985318] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9557.985847] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9562.296727] Lustre: server umount lustre-MDT0000 complete [ 9564.018689] LustreError: 302873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777003773 with bad export cookie 14719561032568142613 [ 9564.019598] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9564.022934] LustreError: 302873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9564.169839] Lustre: server umount lustre-MDT0001 complete [ 9565.691040] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9566.439137] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:09:34 (1777003774) [ 9569.131729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9569.380216] LustreError: 306786:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9569.392845] LustreError: 306786:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 9569.430934] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9571.396977] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9574.881096] LustreError: 306805:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9575.389081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9577.209946] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9579.572110] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9581.246552] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9584.263145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9587.168139] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9589.572860] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9592.795760] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9592.865745] Lustre: lustre-OST0001: new disk, initializing [ 9592.868147] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9592.911454] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9592.914925] Lustre: Skipped 2 previous similar messages [ 9595.035875] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9595.040480] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9595.055424] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9595.487129] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9596.066616] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9596.071549] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9597.091449] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9597.098617] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9599.139588] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9599.144037] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9600.348358] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9600.476389] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9602.110149] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9061a091c000.ost_server_uuid 50 [ 9602.781418] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9061a091c000.ost_server_uuid in FULL state after 0 sec [ 9603.234679] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9603.241718] LustreError: 309549:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 9604.442608] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9061a091c000.ost_server_uuid 50 [ 9605.204870] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9061a091c000.ost_server_uuid in FULL state after 0 sec [ 9606.376616] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9619.354045] Lustre: lustre-OST0001: Not available for connect from 192.168.202.43@tcp (stopping) [ 9619.356591] Lustre: Skipped 1 previous similar message [ 9619.656688] Lustre: server umount lustre-OST0001 complete [ 9620.964240] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9620.965588] LustreError: 308379:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9620.969221] Lustre: Skipped 1 previous similar message [ 9621.987501] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 9621.989976] LustreError: Skipped 1 previous similar message [ 9626.083813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9626.084085] LustreError: 308379:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9626.084591] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9626.084597] Lustre: Skipped 1 previous similar message [ 9626.086052] Lustre: Skipped 1 previous similar message [ 9626.100261] LustreError: 308379:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 9627.747856] Lustre: server umount lustre-OST0000 complete [ 9635.414345] Lustre: server umount lustre-MDT0000 complete [ 9636.321073] LustreError: 306787:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9636.327709] LustreError: 306787:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 9636.892283] LustreError: 309280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777003846 with bad export cookie 14719561032568143495 [ 9636.896019] LustreError: 309280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9636.896167] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9637.026937] Lustre: server umount lustre-MDT0001 complete [ 9641.116338] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 9642.375965] Key type lgssc unregistered [ 9642.531552] LNet: 311719:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9642.536229] LNetError: 311719:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9642.549333] LNet: Removed LNI 192.168.202.143@tcp [ 9642.926871] Key type .llcrypt unregistered [ 9642.928335] Key type ._llcrypt unregistered [ 9652.757424] Key type ._llcrypt registered [ 9652.758388] Key type .llcrypt registered [ 9652.797259] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 9653.143865] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9653.150946] alg: No test for adler32 (adler32-zlib) [ 9654.026377] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 9654.122670] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 9655.712141] Key type lgssc registered [ 9656.138833] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9659.101435] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9659.109200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9660.307679] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9661.986490] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9664.125220] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9667.122167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9667.307445] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9668.933728] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9672.888255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9673.030531] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9675.273511] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9677.159537] LustreError: 314649:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9677.169403] LustreError: 314649:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 9677.689368] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9680.511797] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9680.601336] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9682.946082] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9685.145808] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9685.743933] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9685.744097] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9692.302912] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9692.413239] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9773.026264] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9773.029188] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9773.034544] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9773.036875] Lustre: Skipped 1 previous similar message [ 9778.150984] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9778.153277] Lustre: Skipped 1 previous similar message [ 9783.268358] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9783.271138] Lustre: Skipped 1 previous similar message [ 9786.336163] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9786.374955] Lustre: server umount lustre-OST0001 complete [ 9788.386386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9788.387364] LustreError: 316084:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9788.387837] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9788.387843] Lustre: Skipped 1 previous similar message [ 9788.389079] LustreError: Skipped 1 previous similar message [ 9788.390911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9788.394625] LustreError: 316084:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 9793.506992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9793.507729] LustreError: 316086:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9793.509441] Lustre: Skipped 2 previous similar messages [ 9793.513890] LustreError: 316086:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 9794.133993] Lustre: server umount lustre-OST0000 complete [ 9795.552824] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9795.555880] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9795.559261] Lustre: Skipped 1 previous similar message [ 9798.626973] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9801.566389] Lustre: server umount lustre-MDT0000 complete [ 9802.828230] LustreError: 313081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777004012 with bad export cookie 3393977749746238325 [ 9802.831495] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9802.831684] LustreError: 313081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9802.964819] Lustre: server umount lustre-MDT0001 complete [ 9806.669035] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 9807.803540] Key type lgssc unregistered [ 9807.952465] LNet: 317646:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9807.955501] LNetError: 317646:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9807.963506] LNet: Removed LNI 192.168.202.143@tcp [ 9808.307092] Key type .llcrypt unregistered [ 9808.308407] Key type ._llcrypt unregistered [ 9828.073099] Key type ._llcrypt registered [ 9828.074437] Key type .llcrypt registered [ 9828.116779] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 9828.483621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9828.506684] alg: No test for adler32 (adler32-zlib) [ 9829.378531] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [ 9829.477940] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 9831.080186] Key type lgssc registered [ 9831.540587] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9835.193853] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9835.200884] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9836.453190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9837.973384] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9841.126027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9841.299282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9842.861627] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9844.043231] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9847.156222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9847.342272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9849.542669] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9851.428627] LustreError: 321361:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9851.436518] LustreError: 321361:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 9852.879467] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9852.972371] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9855.172692] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9861.108614] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9861.116044] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9865.172305] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9872.458735] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9881.574358] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9881.575832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9881.578600] Lustre: Skipped 2 previous similar messages [ 9881.582243] Lustre: Skipped 1 previous similar message [ 9886.690030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9886.695143] Lustre: Skipped 3 previous similar messages [ 9887.719156] Lustre: server umount lustre-MDT0000 complete [ 9889.440577] LustreError: 319885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777004098 with bad export cookie 16705934848530607741 [ 9889.444438] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9889.444919] LustreError: 319885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9889.577226] Lustre: server umount lustre-MDT0001 complete [ 9901.003699] Lustre: server umount lustre-OST0000 complete [ 9912.826421] Lustre: server umount lustre-OST0001 complete [ 9919.117508] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 9922.718428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9922.862700] LustreError: 325081:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9922.882893] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9924.372739] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9927.555859] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9929.094503] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9930.147337] Lustre: 326189:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9932.583316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9932.747277] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9932.750233] Lustre: Skipped 1 previous similar message [ 9933.797393] LustreError: 326723:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9933.799830] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9934.921320] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9936.870382] LustreError: 326723:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9936.875736] LustreError: 326723:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 9938.081909] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9940.211781] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 9943.530287] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9943.534465] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9944.813809] Lustre: 328023:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9953.249110] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9953.252306] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9953.256918] Lustre: Skipped 1 previous similar message [ 9953.259246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9957.085146] Lustre: server umount lustre-MDT0000 complete [ 9958.392081] LustreError: 325807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777004167 with bad export cookie 16705934848530609435 [ 9958.395195] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9958.397559] LustreError: 325807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9958.503665] Lustre: server umount lustre-MDT0001 complete [ 9970.186630] Lustre: server umount lustre-OST0000 complete [ 9981.853964] Lustre: server umount lustre-OST0001 complete [ 9983.214720] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9983.813986] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9984.505562] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:16:32 (1777004192) [ 9992.435645] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 9995.416612] loop0: detected capacity change from 0 to 1099511627776 [ 9995.560395] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9995.563082] *** VALIDATE xfs *** [ 9995.570822] XFS (loop0): Mounting V5 Filesystem [ 9995.578424] XFS (loop0): Ending clean mount [ 9996.019800] loop1: detected capacity change from 0 to 18691697672192 [10018.193399] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [10021.600243] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [10034.421663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10034.508502] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10034.521350] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10034.560120] Lustre: lustre-MDT0000: new disk, initializing [10034.586211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10034.588357] Lustre: Skipped 1 previous similar message [10034.593949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10035.899699] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10040.145141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10040.190117] Lustre: 333084:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10040.214903] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10040.216693] Lustre: Skipped 1 previous similar message [10040.252167] Lustre: lustre-MDT0001: new disk, initializing [10040.293742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10040.296833] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10041.607988] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10044.831778] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10046.095275] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10048.456322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10048.558728] Lustre: lustre-OST0000: new disk, initializing [10048.561327] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10050.109201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10050.114563] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10050.147321] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10050.496419] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10053.738445] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10055.168292] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10055.240603] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10056.638921] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10056.722514] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10060.258720] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10060.258915] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10060.260353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10060.260359] Lustre: Skipped 3 previous similar messages [10060.261654] LustreError: Skipped 1 previous similar message [10060.270549] Lustre: Skipped 4 previous similar messages [10063.509229] Lustre: server umount lustre-OST0000 complete [10065.377276] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10065.377944] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10065.380219] Lustre: Skipped 1 previous similar message [10065.383034] Lustre: Skipped 2 previous similar messages [10075.616860] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10075.618621] Lustre: Skipped 3 previous similar messages [10079.200088] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10079.258929] Lustre: server umount lustre-MDT0000 complete [10080.578466] LustreError: 333076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777004289 with bad export cookie 16705934848530610989 [10080.582495] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10080.582919] LustreError: 333076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10080.706282] Lustre: server umount lustre-MDT0001 complete [10083.465165] XFS (loop0): Unmounting Filesystem [10085.005188] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:18:13 (1777004293) [10087.200577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10087.343690] LustreError: 337233:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10087.366642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10087.369137] Lustre: Skipped 2 previous similar messages [10088.576492] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10091.275987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10092.659576] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10094.400487] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10095.572901] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10097.766230] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10099.680335] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10101.413142] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10121.696799] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10121.699498] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10121.703783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10126.229272] Lustre: server umount lustre-OST0000 complete [10132.449484] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10132.453169] LustreError: Skipped 1 previous similar message [10132.455256] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10132.460276] Lustre: Skipped 2 previous similar messages [10133.653286] Lustre: server umount lustre-MDT0000 complete [10134.845579] LustreError: 337215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777004344 with bad export cookie 16705934848530611871 [10134.848085] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10134.849250] LustreError: 337215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10134.966936] Lustre: server umount lustre-MDT0001 complete [10138.495671] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [10139.464985] Key type lgssc unregistered [10139.588359] LNet: 340526:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10139.590920] LNetError: 340526:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10139.596328] LNet: Removed LNI 192.168.202.143@tcp [10139.877096] Key type .llcrypt unregistered [10139.877991] Key type ._llcrypt unregistered [10149.979575] Key type ._llcrypt registered [10149.980616] Key type .llcrypt registered [10150.021770] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [10150.577778] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10150.595784] alg: No test for adler32 (adler32-zlib) [10151.455593] Lustre: Lustre: Build Version: 2.17.52_54_gb645d1c [10151.544603] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [10153.120138] Key type lgssc registered [10153.480672] Lustre: Echo OBD driver; http://www.lustre.org/ [10156.035770] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10156.043561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10157.198332] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10158.453605] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10161.304334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10161.423227] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10162.722634] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10164.568107] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10165.816278] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10168.245744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10168.374095] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10170.411940] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10172.407151] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10187.232862] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10187.236214] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10187.239905] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10187.746917] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10187.750631] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10191.762904] Lustre: server umount lustre-MDT0001 complete [10192.866559] LustreError: 342773:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10192.872761] LustreError: 342773:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [10195.805070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10195.855625] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10195.858166] Lustre: Found index 1 for lustre-MDT0001, updating log [10195.871611] Lustre: 342042:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10196.001982] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10197.580607] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10201.062068] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10202.599151] LustreError: 345427:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10202.602536] LustreError: 345427:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10202.606593] LustreError: 342044:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10202.625314] LustreError: 345430:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.143@tcp: cfg command failed: rc = -17 [10202.628202] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10202.628202] [10503.138518] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10503.138674] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10503.139386] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10503.139390] Lustre: Skipped 1 previous similar message [10503.141381] Lustre: Skipped 1 previous similar message [10505.621952] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.43@tcp (stopping) [10505.623624] Lustre: Skipped 1 previous similar message [10507.027696] Lustre: server umount lustre-MDT0001 complete [10508.257186] LustreError: 342051:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10508.261225] LustreError: 342051:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [10510.269982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10510.299941] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10510.301559] Lustre: Found index 1 for lustre-MDT0001, updating log [10510.313615] Lustre: 342041:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10510.410499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10511.592601] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10515.429429] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10516.455520] LustreError: 357944:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10516.457760] LustreError: 357944:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10516.460639] LustreError: 342044:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10516.464980] LustreError: 342044:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10516.486221] LustreError: 357953:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [10516.488461] LustreError: 357953:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [10516.489933] LustreError: 357953:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.143@tcp: cfg command failed: rc = -17 [10516.492822] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.143@tcp [10516.492822] [10817.506949] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10817.507033] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10817.511217] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10817.514509] Lustre: Skipped 1 previous similar message [10827.745963] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10827.747784] Lustre: Skipped 5 previous similar messages [10828.256110] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10828.307723] Lustre: server umount lustre-MDT0001 complete [10831.692617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10831.722798] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10831.725436] Lustre: Found index 1 for lustre-MDT0001, updating log [10831.736733] Lustre: 342042:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10831.844030] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10833.217351] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [10836.964980] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10840.040386] LustreError: 370740:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10840.043124] LustreError: 370740:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10840.046185] LustreError: 342044:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10840.048955] LustreError: 342044:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10840.065807] LustreError: 370743:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10840.071504] LustreError: 370743:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.143@tcp: cfg command failed: rc = -114 [10840.074952] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10840.074952] [11149.280996] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11149.282747] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11149.285952] Lustre: Skipped 1 previous similar message [11149.287322] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11149.288773] Lustre: Skipped 1 previous similar message [11162.080123] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11162.142144] Lustre: server umount lustre-MDT0000 complete [11162.593985] LustreError: 342047:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11162.600424] LustreError: 342047:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [11163.355227] LustreError: 342767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777005372 with bad export cookie 4553217636494315589 [11163.359227] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11163.360040] LustreError: 342767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11167.712867] LustreError: 344210:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11167.713130] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11167.724668] Lustre: Skipped 2 previous similar messages [11168.736949] LustreError: 343568:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11169.521350] Lustre: server umount lustre-MDT0001 complete [11170.913780] Lustre: server umount lustre-OST0000 complete [11173.878637] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [11177.336836] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [11189.411692] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 00:36:37 (1777005397) [11191.472639] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11191.553540] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11191.562892] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11191.595095] Lustre: lustre-MDT0000: new disk, initializing [11191.617695] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11191.622772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11192.810534] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [11196.755311] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11196.795706] Lustre: 385320:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11196.817577] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11196.819510] Lustre: Skipped 1 previous similar message [11196.857702] Lustre: lustre-MDT0001: new disk, initializing [11196.885828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11196.898890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11196.901945] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11198.080679] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [11200.953176] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11202.040323] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11204.116061] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11204.223789] Lustre: lustre-OST0000: new disk, initializing [11204.225791] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11204.255183] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11205.942775] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [11208.824323] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11210.228020] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11210.230490] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11210.258507] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11217.376728] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11217.378897] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11217.383928] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11217.386709] Lustre: Skipped 8 previous similar messages [11222.677365] Lustre: server umount lustre-MDT0000 complete [11223.824153] LustreError: 385311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777005433 with bad export cookie 4553217636494317528 [11223.826518] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11223.830695] LustreError: 385311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11223.930505] Lustre: server umount lustre-MDT0001 complete [11235.294449] Lustre: server umount lustre-OST0000 complete [11244.680632] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [11247.905905] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [11258.481737] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10885 sec ========= 00:37:46 (1777005466) [11259.057910] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 00:37:47 (1777005467) === [11260.180459] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 00:37:48 (1777005468) === [11272.185291] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [11273.171332] Key type lgssc unregistered [11273.299315] LNet: 393250:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11273.301282] LNetError: 393250:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11273.309338] LNet: Removed LNI 192.168.202.143@tcp [11273.601908] Key type .llcrypt unregistered [11273.602818] Key type ._llcrypt unregistered