[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 477037839 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002428] x2apic enabled [ 0.003011] Switched APIC routing to physical x2apic. [ 0.004015] kvm-guest: setup PV IPIs [ 0.007768] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010159] LSM: Security Framework initializing [ 0.011062] Yama: becoming mindful. [ 0.012037] SELinux: Initializing. [ 0.013085] *** VALIDATE selinux *** [ 0.020423] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024552] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025118] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026082] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028042] *** VALIDATE tmpfs *** [ 0.029492] *** VALIDATE proc *** [ 0.031044] *** VALIDATE cgroup *** [ 0.032007] *** VALIDATE cgroup2 *** [ 0.033265] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034169] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036032] Spectre V2 : User space: Vulnerable [ 0.037011] Speculative Store Bypass: Vulnerable [ 0.040765] debug: unmapping init [mem 0xffffffff92259000-0xffffffff92260fff] [ 0.042179] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043736] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044025] ... version: 2 [ 0.045015] ... bit width: 48 [ 0.046011] ... generic registers: 4 [ 0.047014] ... value mask: 0000ffffffffffff [ 0.048009] ... max period: 00007fffffffffff [ 0.049014] ... fixed-purpose events: 3 [ 0.050014] ... event mask: 000000070000000f [ 0.051328] rcu: Hierarchical SRCU implementation. [ 0.053445] smp: Bringing up secondary CPUs ... [ 0.054411] x86: Booting SMP configuration: [ 0.055031] .... node #0, CPUs: #1 #2 #3 [ 0.058011] smp: Brought up 1 node, 4 CPUs [ 0.059866] smpboot: Max logical packages: 1 [ 0.060014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.136244] node 0 deferred pages initialised in 74ms [ 0.138119] devtmpfs: initialized [ 0.139258] x86/mm: Memory block size: 128MB [ 0.141754] gcov: version magic: 0x41383552 [ 0.143142] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.144109] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.145285] pinctrl core: initialized pinctrl subsystem [ 0.146204] [ 0.146822] ************************************************************* [ 0.147014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.148014] ** ** [ 0.149014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150013] ** ** [ 0.151014] ** This means that this kernel is built to expose internal ** [ 0.152014] ** IOMMU data structures, which may compromise security on ** [ 0.153014] ** your system. ** [ 0.154012] ** ** [ 0.155014] ** If you see this message and you are not debugging the ** [ 0.156017] ** kernel, report this immediately to your vendor! ** [ 0.157014] ** ** [ 0.158017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.159016] ************************************************************* [ 0.160676] NET: Registered protocol family 16 [ 0.161473] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.162071] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.163067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.164459] cpuidle: using governor menu [ 0.166613] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.167455] PCI: Using configuration type 1 for base access [ 0.168134] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.175117] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.176043] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.178097] cryptd: max_cpu_qlen set to 1000 [ 0.180274] ACPI: Added _OSI(Module Device) [ 0.181014] ACPI: Added _OSI(Processor Device) [ 0.182014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.183021] ACPI: Added _OSI(Processor Aggregator Device) [ 0.186866] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.189556] ACPI: Interpreter enabled [ 0.190069] ACPI: PM: (supports S0 S3 S4 S5) [ 0.191016] ACPI: Using IOAPIC for interrupt routing [ 0.192104] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.193408] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.202725] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.203055] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.204021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.205093] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.207086] acpiphp: Slot [2] registered [ 0.208126] acpiphp: Slot [5] registered [ 0.209094] acpiphp: Slot [6] registered [ 0.210150] acpiphp: Slot [7] registered [ 0.211044] acpiphp: Slot [8] registered [ 0.212121] acpiphp: Slot [9] registered [ 0.213023] acpiphp: Slot [10] registered [ 0.213986] acpiphp: Slot [3] registered [ 0.214063] acpiphp: Slot [4] registered [ 0.215089] acpiphp: Slot [11] registered [ 0.216062] acpiphp: Slot [12] registered [ 0.217106] acpiphp: Slot [13] registered [ 0.218079] acpiphp: Slot [14] registered [ 0.219127] acpiphp: Slot [15] registered [ 0.220094] acpiphp: Slot [16] registered [ 0.221101] acpiphp: Slot [17] registered [ 0.222108] acpiphp: Slot [18] registered [ 0.223128] acpiphp: Slot [19] registered [ 0.224122] acpiphp: Slot [20] registered [ 0.225102] acpiphp: Slot [21] registered [ 0.226094] acpiphp: Slot [22] registered [ 0.227171] acpiphp: Slot [23] registered [ 0.228099] acpiphp: Slot [24] registered [ 0.229109] acpiphp: Slot [25] registered [ 0.230146] acpiphp: Slot [26] registered [ 0.231090] acpiphp: Slot [27] registered [ 0.232105] acpiphp: Slot [28] registered [ 0.233098] acpiphp: Slot [29] registered [ 0.234100] acpiphp: Slot [30] registered [ 0.235101] acpiphp: Slot [31] registered [ 0.236072] PCI host bridge to bus 0000:00 [ 0.237020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.238022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.239023] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.240026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.241034] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.242043] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.243174] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.245013] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.247408] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.253015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.256065] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.257024] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.258019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.259025] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.260589] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.261766] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.262047] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.263824] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.266019] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.275018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.278017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.281990] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.284020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.287020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.294021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.299876] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.302012] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.305022] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.312017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.317599] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.320014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.323017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.330020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.335266] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.347021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.359015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.381021] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.397537] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.404015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.410018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.441018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.448214] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.452017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.456015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.467015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.478726] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.482661] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.485640] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.488394] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.490234] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.496447] iommu: Default domain type: Passthrough [ 0.498457] SCSI subsystem initialized [ 0.500160] ACPI: bus type USB registered [ 0.501125] usbcore: registered new interface driver usbfs [ 0.503106] usbcore: registered new interface driver hub [ 0.504226] usbcore: registered new device driver usb [ 0.506126] pps_core: LinuxPPS API ver. 1 registered [ 0.508014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.511088] PTP clock support registered [ 0.513038] EDAC MC: Ver: 3.0.0 [ 0.514818] PCI: Using ACPI for IRQ routing [ 0.517593] NetLabel: Initializing [ 0.519009] NetLabel: domain hash size = 128 [ 0.520009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.522080] NetLabel: unlabeled traffic allowed by default [ 0.524323] vgaarb: loaded [ 0.526183] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.527010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.535180] clocksource: Switched to clocksource kvm-clock [ 0.635887] VFS: Disk quotas dquot_6.6.0 [ 0.636919] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.638578] *** VALIDATE ramfs *** [ 0.639695] *** VALIDATE hugetlbfs *** [ 0.640599] pnp: PnP ACPI init [ 0.642074] pnp: PnP ACPI: found 6 devices [ 0.658785] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.661539] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.663595] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.665669] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.667943] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.670224] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.672995] NET: Registered protocol family 2 [ 0.675749] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.680969] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.684359] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.689360] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.693360] TCP: Hash tables configured (established 65536 bind 65536) [ 0.696349] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.699451] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.702277] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.705292] NET: Registered protocol family 1 [ 0.707690] RPC: Registered named UNIX socket transport module. [ 0.709357] RPC: Registered udp transport module. [ 0.710393] RPC: Registered tcp transport module. [ 0.711806] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.713373] NET: Registered protocol family 44 [ 0.714441] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.715764] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.717287] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.719238] PCI: CLS 0 bytes, default 64 [ 0.720880] Unpacking initramfs... [ 2.065790] debug: unmapping init [mem 0xffffa0017cc54000-0xffffa0017ffbffff] [ 2.069649] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.071773] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.074107] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.566700] Initialise system trusted keyrings [ 2.568265] Key type blacklist registered [ 2.570604] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.579037] zbud: loaded [ 2.582141] *** VALIDATE nfs *** [ 2.583478] *** VALIDATE nfs4 *** [ 2.585452] pstore: using deflate compression [ 2.590307] Platform Keyring initialized [ 2.672463] NET: Registered protocol family 38 [ 2.673911] Key type asymmetric registered [ 2.674844] Asymmetric key parser 'x509' registered [ 2.676229] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.678215] io scheduler mq-deadline registered [ 2.679918] io scheduler kyber registered [ 2.681537] io scheduler bfq registered [ 2.683774] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.686274] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.689073] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.691950] ACPI: Power Button [PWRF] [ 2.697659] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.703310] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.719787] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.729790] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.742728] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.770596] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.798620] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.803283] Non-volatile memory driver v1.3 [ 2.804936] Linux agpgart interface v0.103 [ 2.830789] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 2.832681] vda: detected capacity change from 0 to 74698752 [ 2.843742] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.845723] vdb: detected capacity change from 0 to 1073741824 [ 2.857560] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.860234] vdc: detected capacity change from 0 to 2621440000 [ 2.877499] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.880668] vdd: detected capacity change from 0 to 2621440000 [ 2.893484] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.896407] vde: detected capacity change from 0 to 4294967296 [ 2.908595] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.911306] vdf: detected capacity change from 0 to 4294967296 [ 2.917264] libphy: Fixed MDIO Bus: probed [ 2.926894] usbcore: registered new interface driver usbserial_generic [ 2.929190] usbserial: USB Serial support registered for generic [ 2.931668] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.935944] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.937805] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.940800] mousedev: PS/2 mouse device common for all mice [ 2.943614] rtc_cmos 00:05: RTC can wake from S4 [ 2.947118] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.947172] rtc_cmos 00:05: registered as rtc0 [ 2.952473] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.955286] intel_pstate: CPU model not supported [ 2.956920] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.962239] hid: raw HID events driver (C) Jiri Kosina [ 2.964544] usbcore: registered new interface driver usbhid [ 2.966696] usbhid: USB HID core driver [ 2.967197] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.968588] drop_monitor: Initializing network drop monitor service [ 2.973916] Initializing XFRM netlink socket [ 2.976256] NET: Registered protocol family 10 [ 2.978769] Segment Routing with IPv6 [ 2.980087] NET: Registered protocol family 17 [ 2.981918] mpls_gso: MPLS GSO support [ 2.988389] RAS: Correctable Errors collector initialized. [ 2.990105] AVX version of gcm_enc/dec engaged. [ 2.991748] AES CTR mode by8 optimization enabled [ 3.065370] sched_clock: Marking stable (3065346016, 0)->(4129798197, -1064452181) [ 3.067824] registered taskstats version 1 [ 3.069512] Loading compiled-in X.509 certificates [ 3.070992] zswap: loaded using pool lzo/zbud [ 3.091669] Key type big_key registered [ 3.101067] Key type encrypted registered [ 3.102261] ima: No TPM chip found, activating TPM-bypass! [ 3.103958] ima: Allocated hash algorithm: sha1 [ 3.105093] ima: No architecture policies found [ 3.106470] evm: Initialising EVM extended attributes: [ 3.107842] evm: security.selinux [ 3.108626] evm: security.ima [ 3.109283] evm: security.capability [ 3.110203] evm: HMAC attrs: 0x1 [ 3.112175] rtc_cmos 00:05: setting system clock to 2026-08-12 20:12:15 UTC (1786565535) [ 3.117644] debug: unmapping init [mem 0xffffffff93203000-0xffffffff933fffff] [ 3.120471] debug: unmapping init [mem 0xffffffff91f82000-0xffffffff92258fff] [ 3.132124] Write protecting the kernel read-only data: 28672k [ 3.135586] debug: unmapping init [mem 0xffffffff90603000-0xffffffff907fffff] [ 3.138395] debug: unmapping init [mem 0xffffffff90f14000-0xffffffff90ffffff] [ 3.172178] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.180762] systemd[1]: Detected virtualization kvm. [ 3.182713] systemd[1]: Detected architecture x86-64. [ 3.184575] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.210152] systemd[1]: No hostname configured. [ 3.211845] systemd[1]: Set hostname to . [ 3.213432] random: systemd: uninitialized urandom read (16 bytes read) [ 3.214934] systemd[1]: Initializing machine ID from random generator. [ 3.332627] random: systemd: uninitialized urandom read (16 bytes read) [ 3.335236] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.339236] random: systemd: uninitialized urandom read (16 bytes read) [ 3.341273] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.344733] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Timers. [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Sockets. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.820947] device-mapper: uevent: version 1.0.3 [ 3.823048] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK [ 4.127809] random: fast init done ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.431224] virtio_net virtio0 ens2: renamed from eth0 [ 4.469215] scsi host0: ata_piix [ 4.472260] scsi host1: ata_piix [ 4.474488] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.477966] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.626200] random: crng init done [ 9.628172] random: 7 urandom warning(s) missed due to ratelimiting [ 9.632371] dracut-initqueue[589]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.464637] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 14.261352] printk: systemd: 26 output lines suppressed due to ratelimiting [ 15.104654] SELinux: Disabled at runtime. [ 15.212845] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.248667] systemd[1]: Detected virtualization kvm. [ 15.263749] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.534603] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.546858] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.568348] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.580069] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.595318] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.612430] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.633390] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Apply Kernel Variables... Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ 18.042476] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 18.984460] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.326220] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 20.447322] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 20.872263] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.003661] EDAC sbridge: Ver: 1.1.2 [ 24.857422] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 25.755416] NFS: Registering the id_resolver key type [ 25.758295] Key type id_resolver registered [ 25.760073] Key type id_legacy registered [*** ] A start job is running for Configur…-only root support (8s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Mark the need to relabel after reboot. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... Starting Login Service... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg135-server login: [ 89.318469] libcfs: loading out-of-tree module taints kernel. [ 89.413857] Key type ._llcrypt registered [ 89.416155] Key type .llcrypt registered [ 89.565897] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 110.626094] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 112.403152] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 112.414863] alg: No test for adler32 (adler32-zlib) [ 113.966168] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 114.796636] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 116.593865] Key type lgssc registered [ 118.655154] Lustre: Echo OBD driver; http://www.lustre.org/ [ 140.573585] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 184.738697] hrtimer: interrupt took 3478712 ns [ 188.582817] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 205.070934] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 205.137778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 206.552201] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 206.608135] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 206.841276] Lustre: lustre-MDT0000: new disk, initializing [ 207.015747] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 207.045703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 214.142360] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 230.413221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 230.641433] Lustre: 6519:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 230.693066] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 230.699920] Lustre: Skipped 1 previous similar message [ 230.840332] Lustre: lustre-MDT0001: new disk, initializing [ 230.954078] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 231.017648] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 231.061274] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 237.362339] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 243.388417] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 254.694859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 255.041248] Lustre: lustre-OST0000: new disk, initializing [ 255.045921] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 255.055079] Lustre: 8458:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 255.121348] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 260.665592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 260.676671] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 260.809402] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 262.741985] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 280.603488] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 280.836340] Lustre: lustre-OST0001: new disk, initializing [ 280.856482] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 280.866614] Lustre: 9532:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 280.980168] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 288.281710] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 289.829435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 289.841942] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 289.931905] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 302.148444] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 313.813759] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 323.034257] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing check_logdir /tmp/testlogs/ [ 328.992686] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing yml_node [ 335.624440] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 339.015409] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 341.876756] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 343.581266] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 12 16:17:53 EDT 2026 [ 363.261032] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 365.144375] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 381.927510] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 381.936881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 381.962316] Lustre: Skipped 2 previous similar messages [ 381.976812] Lustre: Skipped 2 previous similar messages [ 387.132291] Lustre: server umount lustre-MDT0000 complete [ 391.461836] LustreError: 6510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565923 with bad export cookie 7985941206720884060 [ 391.474037] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 392.162861] LustreError: 8253:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 392.166386] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 392.190534] LustreError: 8253:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 392.216756] Lustre: Skipped 1 previous similar message [ 392.227865] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 392.241627] Lustre: Skipped 2 previous similar messages [ 396.192871] LustreError: 6524:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 396.203341] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 396.207206] LustreError: 6524:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 396.221063] Lustre: Skipped 1 previous similar message [ 398.077616] Lustre: server umount lustre-MDT0001 complete [ 402.612960] Lustre: server umount lustre-OST0000 complete [ 407.071384] Lustre: server umount lustre-OST0001 complete [ 416.135161] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 426.405416] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 462.312824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 462.588097] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 462.630738] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 462.709569] Lustre: lustre-MDT0000: new disk, initializing [ 462.789301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 462.809146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 466.980300] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 477.843664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 477.938973] Lustre: 15086:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 477.979791] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 477.982530] Lustre: Skipped 1 previous similar message [ 478.044205] Lustre: lustre-MDT0001: new disk, initializing [ 478.136514] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 478.182747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 478.201803] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 482.633824] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 491.714549] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 497.781922] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 505.116983] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 505.569955] Lustre: lustre-OST0000: new disk, initializing [ 505.575774] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 505.585344] Lustre: 17056:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 505.734910] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 507.555328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 507.568238] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 507.777475] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 512.281565] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 523.212696] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 527.780045] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 527.990739] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 533.325636] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 533.637684] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 538.596596] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 538.598238] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 538.642894] LustreError: Skipped 1 previous similar message [ 538.646455] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 538.670482] Lustre: Skipped 2 previous similar messages [ 538.717018] Lustre: Skipped 1 previous similar message [ 543.723655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 543.741133] Lustre: Skipped 1 previous similar message [ 550.880235] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 551.095746] Lustre: server umount lustre-OST0000 complete [ 559.075706] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 559.079171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 559.095305] Lustre: Skipped 1 previous similar message [ 559.109491] Lustre: Skipped 3 previous similar messages [ 562.668062] Lustre: server umount lustre-MDT0000 complete [ 564.197939] LustreError: 15094:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 564.222045] LustreError: 15094:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 566.534595] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566098 with bad export cookie 7985941206720885593 [ 566.539102] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 566.555047] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 567.129617] Lustre: server umount lustre-MDT0001 complete [ 572.102853] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:21:42 (1786566102) [ 579.458791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 580.358990] LustreError: 19323:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 580.430228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 585.552366] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 585.696805] LustreError: 19324:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 593.610408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 597.901905] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 605.753393] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 612.477635] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 619.117565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 625.825680] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 634.603963] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 638.901891] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 650.221235] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 650.234226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 650.244119] Lustre: Skipped 2 previous similar messages [ 650.258719] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 650.262723] Lustre: Skipped 1 previous similar message [ 652.459355] Lustre: server umount lustre-MDT0000 complete [ 655.337100] LustreError: 19319:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 655.364490] LustreError: 19319:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 656.518112] LustreError: 19305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566188 with bad export cookie 7985941206720886475 [ 656.543577] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 657.042695] Lustre: server umount lustre-MDT0001 complete [ 670.903131] Lustre: server umount lustre-OST0000 complete [ 690.773299] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 702.612771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 703.128737] LustreError: 23704:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 703.217095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 703.220754] Lustre: Skipped 2 previous similar messages [ 708.084190] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 716.780880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 721.782703] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 724.987323] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 732.256373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 739.948706] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 748.839212] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 748.978469] Lustre: lustre-OST0001: new disk, initializing [ 748.981644] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 748.986675] Lustre: 26021:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 750.187661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 750.192531] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 750.322493] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 755.319990] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 764.658450] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 768.301488] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 781.283118] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 781.297840] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 781.305074] Lustre: Skipped 3 previous similar messages [ 781.329020] Lustre: Skipped 4 previous similar messages [ 787.221426] Lustre: server umount lustre-MDT0000 complete [ 790.743316] LustreError: 23686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566323 with bad export cookie 7985941206720887322 [ 790.745617] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 790.753585] LustreError: 23686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 791.270621] Lustre: server umount lustre-MDT0001 complete [ 805.569835] Lustre: server umount lustre-OST0000 complete [ 806.881670] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786566323/real 1786566323] req@ffffa001f7a0f100 x1873349860121216/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786566339 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 806.913915] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 808.945973] Lustre: server umount lustre-OST0001 complete [ 824.384715] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 832.960186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 833.292956] LustreError: 29040:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 833.307139] LustreError: 29040:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 833.378617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 833.384514] Lustre: Skipped 3 previous similar messages [ 837.770300] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 845.913426] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 851.210371] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 854.354658] Lustre: 30188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 860.463945] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 866.782988] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 875.088305] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 881.645750] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 888.750799] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 891.947619] Lustre: 32065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 899.042410] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 899.048286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 899.073028] Lustre: Skipped 3 previous similar messages [ 899.097976] Lustre: Skipped 7 previous similar messages [ 904.036802] Lustre: server umount lustre-MDT0000 complete [ 904.178018] LustreError: 30569:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 904.188319] LustreError: 30569:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 908.098771] LustreError: 29022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566440 with bad export cookie 7985941206720888617 [ 908.104367] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 908.110439] LustreError: 29022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 908.384973] Lustre: server umount lustre-MDT0001 complete [ 922.656549] Lustre: server umount lustre-OST0000 complete [ 924.645155] Lustre: 3650:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786566441/real 1786566441] req@ffffa000c36a5f80 x1873349860167680/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786566457 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 926.243401] Lustre: server umount lustre-OST0001 complete [ 934.699190] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:27:45 (1786566465) [ 964.246036] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 974.784579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 980.589270] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 990.422045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 995.424108] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 998.006188] Lustre: 36019:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1004.171444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1010.559327] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1017.986672] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1024.697934] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1031.648958] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1035.549655] Lustre: 37897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1040.875440] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1040.881878] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1040.890817] Lustre: Skipped 3 previous similar messages [ 1040.903366] Lustre: Skipped 3 previous similar messages [ 1047.379480] Lustre: server umount lustre-MDT0000 complete [ 1051.123580] LustreError: 34861:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1051.163553] LustreError: 34861:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 1051.632496] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566584 with bad export cookie 7985941206720889723 [ 1051.650703] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1051.662266] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1052.256395] Lustre: server umount lustre-MDT0001 complete [ 1066.581102] Lustre: server umount lustre-OST0000 complete [ 1080.594377] Lustre: server umount lustre-OST0001 complete [ 1089.434213] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:30:19 (1786566619) [ 1106.353885] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1116.892174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1117.421454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1117.427562] Lustre: Skipped 7 previous similar messages [ 1122.354795] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1130.448420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1136.681994] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1139.842133] Lustre: 41189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1146.610184] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1154.265772] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1163.068885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1170.419347] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1178.339549] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1182.551711] Lustre: 43064:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1183.550543] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1183.558134] Lustre: Skipped 1 previous similar message [ 1187.301707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1187.306157] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1187.313398] Lustre: Skipped 1 previous similar message [ 1192.835648] Lustre: server umount lustre-MDT0000 complete [ 1196.510506] LustreError: 40020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566728 with bad export cookie 7985941206720890829 [ 1196.522301] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1196.528859] LustreError: 40020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1202.942949] Lustre: server umount lustre-MDT0001 complete [ 1211.240192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1221.098198] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x6ed3c33757c0cc34 [ 1221.112089] Lustre: MGC192.168.201.135@tcp: Connection restored to 0@lo (at 0@lo) [ 1226.281927] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1230.759588] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1234.094760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1239.241983] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1240.038630] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1240.054169] Lustre: Skipped 1 previous similar message [ 1246.540494] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1248.239923] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1253.584643] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1255.069437] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1260.006613] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1265.564849] Lustre: server umount lustre-MDT0000 complete [ 1269.108138] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566801 with bad export cookie 7985941206720891956 [ 1269.108736] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1269.116843] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1269.544980] Lustre: server umount lustre-MDT0001 complete [ 1282.865924] Lustre: server umount lustre-OST0000 complete [ 1296.517653] Lustre: server umount lustre-OST0001 complete [ 1305.971783] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:33:56 (1786566836) [ 1323.027832] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1333.340726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1333.721121] LustreError: 47671:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1333.732342] LustreError: 47671:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 28 previous similar messages [ 1337.800875] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1346.936445] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1353.248393] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1356.627995] Lustre: 48825:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1364.071648] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1370.895433] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1379.499366] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1385.891904] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1392.913957] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1397.067064] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1410.533075] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1410.547731] Lustre: Skipped 19 previous similar messages [ 1424.352224] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1424.590722] Lustre: server umount lustre-MDT0000 complete [ 1428.789114] LustreError: 47652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566961 with bad export cookie 7985941206720892747 [ 1428.794696] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1428.808968] LustreError: 47652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1429.232184] Lustre: server umount lustre-MDT0001 complete [ 1445.273806] Lustre: server umount lustre-OST0000 complete [ 1447.585660] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786566963/real 1786566963] req@ffffa001f56f6680 x1873349860361344/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786566979 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1447.649903] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1447.697582] Lustre: Skipped 13 previous similar messages [ 1452.513710] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786566968/real 1786566968] req@ffffa001f56f4700 x1873349860361856/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786566984 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1452.597835] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1453.714322] Lustre: server umount lustre-OST0001 complete [ 1472.076412] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:36:41 (1786567001) [ 1475.930249] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1478.438620] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:36:48 (1786567008) [ 1491.855263] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 1501.862398] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1529.036248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1529.290365] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1529.297995] Lustre: Skipped 1 previous similar message [ 1529.335721] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1529.426891] Lustre: lustre-MDT0000: new disk, initializing [ 1529.567838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1535.746786] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1546.313595] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1554.695657] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1555.090537] Lustre: lustre-OST0000: new disk, initializing [ 1555.094599] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1555.099562] Lustre: Skipped 1 previous similar message [ 1555.111546] Lustre: 54904:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1557.144848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1557.160559] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1557.215168] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1562.518710] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1573.399660] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1576.326089] Lustre: DEBUG MARKER: create test files [ 1615.309314] Lustre: server umount lustre-OST0000 complete [ 1619.712122] Lustre: server umount lustre-MDT0000 complete [ 1630.866703] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1633.772057] Key type lgssc unregistered [ 1634.072234] LNet: 56538:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1634.084330] LNetError: 56538:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1634.095319] LNet: Removed LNI 192.168.201.135@tcp [ 1635.158256] Key type .llcrypt unregistered [ 1635.162138] Key type ._llcrypt unregistered [ 1666.642366] Key type ._llcrypt registered [ 1666.644444] Key type .llcrypt registered [ 1666.752634] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1668.026077] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1668.076527] alg: No test for adler32 (adler32-zlib) [ 1669.094654] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 1669.309858] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1670.928150] Key type lgssc registered [ 1672.004767] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1679.360106] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1679.372255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1680.778524] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1685.505667] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1692.959578] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1703.001192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1703.668231] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1705.718991] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1705.853466] Lustre: 59274:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1705.866626] Lustre: 59274:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1705.877788] Lustre: 59274:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1705.889904] Lustre: 59274:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1705.898034] Lustre: 59274:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1705.912936] Lustre: 59274:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1711.330201] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1721.023405] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1728.617944] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1736.161926] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1771.019385] Lustre: server umount lustre-OST0000 complete [ 1786.551938] Lustre: server umount lustre-MDT0000 complete [ 1797.201861] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1800.084365] Key type lgssc unregistered [ 1800.367301] LNet: 60913:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1800.377050] LNetError: 60913:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1801.455585] LNet: Removed LNI 192.168.201.135@tcp [ 1802.302923] Key type .llcrypt unregistered [ 1802.308504] Key type ._llcrypt unregistered [ 1836.360833] Key type ._llcrypt registered [ 1836.362817] Key type .llcrypt registered [ 1836.499880] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1838.573961] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1838.618837] alg: No test for adler32 (adler32-zlib) [ 1839.564230] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 1839.711091] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1841.344615] Key type lgssc registered [ 1842.610968] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1850.527587] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1850.550424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1852.167592] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1857.259629] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1867.010564] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1875.781843] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1876.363758] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1878.416464] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1878.542619] Lustre: 63750:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1878.550515] Lustre: 63750:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1878.558411] Lustre: 63750:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1878.565260] Lustre: 63750:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1878.578013] Lustre: 63750:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1878.584564] Lustre: 63750:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1884.496093] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1893.665690] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1899.836686] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1909.219907] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1909.236573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1914.352592] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1915.374877] Lustre: server umount lustre-OST0000 complete [ 1919.716570] Lustre: server umount lustre-MDT0000 complete [ 1931.971689] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1934.886889] Key type lgssc unregistered [ 1935.290830] LNet: 65312:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1935.301517] LNetError: 65312:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1936.371235] LNet: Removed LNI 192.168.201.135@tcp [ 1937.385347] Key type .llcrypt unregistered [ 1937.390548] Key type ._llcrypt unregistered [ 1957.487578] Key type ._llcrypt registered [ 1957.489293] Key type .llcrypt registered [ 1957.585851] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 1974.174471] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1975.675976] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1975.905280] alg: No test for adler32 (adler32-zlib) [ 1976.944245] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 1977.223967] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1978.961245] Key type lgssc registered [ 1980.476527] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2019.547858] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2019.576309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2020.912422] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2020.931561] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2020.991234] Lustre: lustre-MDT0000: new disk, initializing [ 2021.064458] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2021.095057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2026.339125] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2037.267270] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2037.421749] Lustre: 68466:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2037.472955] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2037.476070] Lustre: Skipped 1 previous similar message [ 2037.524205] Lustre: lustre-MDT0001: new disk, initializing [ 2037.598484] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2037.631182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2037.641934] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2042.520454] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2052.929773] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2058.566263] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2065.342383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2065.649979] Lustre: lustre-OST0000: new disk, initializing [ 2065.660561] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2065.675646] Lustre: 70436:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2065.796038] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2071.892896] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2072.103086] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2072.113808] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2072.226139] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2082.447252] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2087.377715] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2087.591440] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2092.026173] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2092.224232] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2097.636108] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2097.644279] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2097.663040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2098.145242] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2098.151603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2103.267282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2103.277132] Lustre: Skipped 2 previous similar messages [ 2108.384744] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2108.398429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2108.407837] Lustre: Skipped 1 previous similar message [ 2108.560729] Lustre: server umount lustre-OST0000 complete [ 2113.505326] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2113.519992] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2113.526126] Lustre: Skipped 1 previous similar message [ 2113.549755] Lustre: Skipped 2 previous similar messages [ 2118.626213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2118.638567] Lustre: Skipped 1 previous similar message [ 2119.435598] Lustre: server umount lustre-MDT0000 complete [ 2123.170046] LustreError: 68461:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786567655 with bad export cookie 16040809467758618089 [ 2123.189307] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2123.520922] Lustre: server umount lustre-MDT0001 complete [ 2132.828555] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:47:43 (1786567663) [ 2144.999810] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2191.321138] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 2201.017086] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2243.114598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2243.320889] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2243.345708] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2243.401542] Lustre: lustre-MDT0000: new disk, initializing [ 2243.464751] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2243.475585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2248.222670] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2258.508051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2258.608787] Lustre: 76454:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2258.675279] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2258.677752] Lustre: Skipped 1 previous similar message [ 2258.737363] Lustre: lustre-MDT0001: new disk, initializing [ 2258.809241] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2258.837177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2258.844606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2263.067881] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2272.105405] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2278.107777] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2285.164440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2285.463823] Lustre: lustre-OST0000: new disk, initializing [ 2285.466918] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2285.472890] Lustre: 78425:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2285.561839] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2291.540349] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2294.817280] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2294.823353] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2294.894470] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2300.669448] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2306.005604] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2306.147341] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2311.600401] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2311.836605] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2315.236216] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2315.239215] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2315.262915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2315.285240] Lustre: Skipped 1 previous similar message [ 2315.309086] Lustre: Skipped 1 previous similar message [ 2328.036196] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2328.191662] Lustre: server umount lustre-OST0000 complete [ 2335.726750] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2335.733470] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2335.736369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2335.736378] Lustre: Skipped 4 previous similar messages [ 2335.785178] Lustre: Skipped 1 previous similar message [ 2338.800373] Lustre: server umount lustre-MDT0000 complete [ 2344.494760] LustreError: 76448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786567876 with bad export cookie 16040809467758618971 [ 2344.500549] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2344.505901] LustreError: 76448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2345.244996] Lustre: server umount lustre-MDT0001 complete [ 2355.866648] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:51:26 (1786567886) [ 2364.740808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2365.332315] LustreError: 80844:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2365.358453] LustreError: 80844:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2365.453078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2370.537228] LustreError: 80845:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2371.216461] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2374.630657] LustreError: 80844:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2379.753129] LustreError: 80845:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2380.819665] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2386.831060] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2395.601252] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2401.816252] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2409.212563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2409.527484] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2409.536707] Lustre: Skipped 1 previous similar message [ 2417.353984] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2426.579493] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2430.278287] Lustre: 83516:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting c37f38ad-4a1e-4573-93fa-b367d6dc9a59 at adminstrative request [ 2430.954223] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2430.964136] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2441.194621] Lustre: MGS: Client c37f38ad-4a1e-4573-93fa-b367d6dc9a59 (at 0@lo) reconnecting [ 2441.199666] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xde9c67391b79accd to 0xde9c67391b79af6d [ 2441.210716] Lustre: MGC192.168.201.135@tcp: Connection restored to 0@lo (at 0@lo) [ 2467.818577] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2467.976783] Lustre: lustre-OST0001: new disk, initializing [ 2467.980079] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2467.984076] Lustre: 83912:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2474.819658] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2476.155022] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2476.167872] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2476.301277] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2485.785338] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2491.873499] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2491.879128] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2491.894867] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2491.897962] Lustre: Skipped 1 previous similar message [ 2491.904483] Lustre: Skipped 2 previous similar messages [ 2495.756660] Lustre: server umount lustre-OST0001 complete [ 2497.011928] LustreError: 84345:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2497.043357] LustreError: 84345:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2507.246123] LustreError: 82667:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2507.263555] LustreError: 82667:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2514.400158] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2514.613992] Lustre: server umount lustre-OST0000 complete [ 2519.528102] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2519.537361] LustreError: Skipped 1 previous similar message [ 2519.553492] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2519.565590] Lustre: Skipped 2 previous similar messages [ 2533.344477] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2533.774921] Lustre: server umount lustre-MDT0000 complete [ 2537.956582] LustreError: 82961:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2537.976672] LustreError: 82961:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2538.270835] LustreError: 80824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568070 with bad export cookie 16040809467758620525 [ 2538.273466] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2538.296979] LustreError: 80824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2538.747199] Lustre: server umount lustre-MDT0001 complete [ 2549.758987] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:54:40 (1786568080) [ 2551.469908] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2553.414544] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:54:43 (1786568083) [ 2555.682827] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2557.975375] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:54:48 (1786568088) [ 2559.591441] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2561.581083] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:54:51 (1786568091) [ 2563.488674] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2565.366246] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:54:55 (1786568095) [ 2567.168409] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2573.861159] loop: module loaded [ 2573.888060] loop0: detected capacity change from 0 to 4294967296 [ 2573.951252] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2575.791498] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2580.286483] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2584.707832] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2612.790599] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2612.827361] LustreError: 88042:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2612.833311] LustreError: 88042:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2612.837860] LustreError: 88042:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2612.847263] LustreError: 88042:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2633.165365] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 2642.054963] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2684.104036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2684.318490] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2684.347760] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2684.467499] Lustre: lustre-MDT0000: new disk, initializing [ 2684.585498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2684.589836] Lustre: Skipped 1 previous similar message [ 2684.607397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2689.640687] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2700.796357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2700.910794] Lustre: 90908:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2701.112293] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2706.116297] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2717.498553] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2725.056070] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2732.743172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2733.079852] Lustre: 92877:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2740.429570] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2740.874772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2741.037114] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2751.395279] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2756.232338] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2756.516858] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2761.379527] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2761.596554] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2766.817449] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2766.820765] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2766.830020] LustreError: Skipped 1 previous similar message [ 2766.839722] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2766.839731] Lustre: Skipped 13 previous similar messages [ 2766.867387] Lustre: Skipped 2 previous similar messages [ 2770.102927] Lustre: server umount lustre-OST0000 complete [ 2789.344260] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2789.521982] Lustre: server umount lustre-MDT0000 complete [ 2792.426182] LustreError: 90919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2792.447675] LustreError: 90919:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2793.422901] LustreError: 90902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568325 with bad export cookie 16040809467758621141 [ 2793.436593] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2793.441082] LustreError: 90902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2793.874434] Lustre: server umount lustre-MDT0001 complete [ 2803.965818] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:58:54 (1786568334) [ 2811.007770] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2816.224579] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2825.242715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2825.550409] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2825.553373] Lustre: Skipped 3 previous similar messages [ 2830.084310] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2837.328904] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2842.427940] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2848.545908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2854.305118] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2862.542247] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2867.431221] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2867.646225] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2872.012340] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2872.261575] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2879.769412] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2880.005666] Lustre: lustre-OST0001: new disk, initializing [ 2880.013735] Lustre: Skipped 2 previous similar messages [ 2880.021811] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2880.026549] Lustre: Skipped 3 previous similar messages [ 2880.037264] Lustre: 98658:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2887.457912] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2887.770233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2887.785029] Lustre: Skipped 2 previous similar messages [ 2887.803626] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2887.918756] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2896.850667] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2900.694405] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2900.878702] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2904.654104] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2904.893081] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2922.919776] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2946.354691] Lustre: Failing over lustre-MDT0000 [ 2946.872696] Lustre: server umount lustre-MDT0000 complete [ 2948.138471] LustreError: 95287:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2948.154422] LustreError: 95287:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2948.580287] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2948.588776] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2948.603897] Lustre: Skipped 2 previous similar messages [ 2965.460946] Lustre: 66691:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786568481/real 1786568482] req@0000000047889338 x1873351812598784/t0(0) o400->MGC192.168.201.135@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786568497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2965.498637] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2971.205791] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2971.208405] LDISKFS-fs (dm-0): recovery complete [ 2971.215403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2975.103503] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2979.282491] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2979.857121] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2979.859208] Lustre: 100597:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2980.337968] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2980.371531] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2980.382396] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 3 previous similar messages [ 2980.457748] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2980.784091] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2981.130445] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2981.144024] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2981.472155] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2981.481255] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2981.495694] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2981.513521] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 5 previous similar messages [ 2982.163458] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2982.181393] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2982.509667] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2982.514341] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2983.784650] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2983.796504] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2984.444208] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2984.459976] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2984.784129] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2984.786747] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2987.977440] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2987.989771] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2988.641255] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2988.653338] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2988.979265] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2988.998533] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2996.282585] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2996.290240] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 2996.929289] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2996.942963] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2997.272801] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2997.279712] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 3012.436107] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3012.454183] Lustre: 101524:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 3013.097459] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3013.112360] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3013.432141] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3013.444228] LustreError: 101524:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3039.026171] Lustre: 101524:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3039.038655] Lustre: 101524:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3039.043205] Lustre: 101524:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client f0635c3e-77f0-4279-b3b0-8611c83df00d@192.168.201.35@tcp [ 3039.050685] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3039.053700] Lustre: 101524:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@0000000040c64d17 x1873351794562432/t0(8589934781) o101->f0635c3e-77f0-4279-b3b0-8611c83df00d@192.168.201.35@tcp:2/0 lens 592/0 e 0 to 0 dl 1786568582 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3039.080716] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3039.102639] Lustre: lustre-MDT0000: Denying connection for new client f0635c3e-77f0-4279-b3b0-8611c83df00d (at 192.168.201.35@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3039.128134] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3039.160575] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3039.165500] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3039.166417] Lustre: Skipped 2 previous similar messages [ 3039.219243] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3039.220609] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3054.561188] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3054.564700] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3054.567837] Lustre: Skipped 7 previous similar messages [ 3067.360299] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3067.490160] Lustre: server umount lustre-OST0000 complete [ 3076.736764] Lustre: server umount lustre-OST0001 complete [ 3080.161136] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3080.169171] Lustre: Skipped 7 previous similar messages [ 3086.145959] Lustre: server umount lustre-MDT0000 complete [ 3089.225758] LustreError: 95271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568621 with bad export cookie 16040809467758672353 [ 3089.235277] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3089.236533] LustreError: 95271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3089.499972] Lustre: server umount lustre-MDT0001 complete [ 3097.367504] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 17:03:47 (1786568627) [ 3104.298526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3104.303666] Lustre: *** cfs_fail_loc=197, val=0*** [ 3119.843227] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3124.964082] LustreError: 103500:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.135@tcp: failed processing log, type 4: rc = -110 [ 3130.099980] Lustre: *** cfs_fail_loc=197, val=0*** [ 3130.101831] Lustre: Skipped 65 previous similar messages [ 3150.688252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3150.695108] Lustre: Skipped 3 previous similar messages [ 3157.005347] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3164.085368] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3166.168946] Lustre: server umount lustre-OST0000 complete [ 3173.554811] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 17:05:04 (1786568704) [ 3192.457084] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3192.794248] LustreError: 105117:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3192.813094] LustreError: 105117:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 46 previous similar messages [ 3196.637101] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3205.531420] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3209.448085] LustreError: 105148:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3209.455453] LustreError: 105148:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3209.459606] LustreError: 105148:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 3209.730377] Lustre: server umount lustre-MDT0000 complete [ 3239.245514] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3247.066576] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3283.308755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3283.563235] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3283.596895] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3283.707269] Lustre: lustre-MDT0000: new disk, initializing [ 3283.788260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3287.908320] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3297.582261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3297.695245] Lustre: 109321:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3298.052227] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3302.254348] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3311.679861] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3316.971977] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3323.689723] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3323.988682] Lustre: 111289:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3329.795176] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3331.244786] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3340.234375] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3344.985468] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3345.133362] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3348.945747] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3349.107559] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3355.638052] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3355.638137] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3355.642295] LustreError: Skipped 3 previous similar messages [ 3355.644853] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3355.660937] Lustre: Skipped 2 previous similar messages [ 3355.672172] Lustre: Skipped 14 previous similar messages [ 3365.344120] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3365.460597] Lustre: server umount lustre-OST0000 complete [ 3374.901708] Lustre: server umount lustre-MDT0000 complete [ 3377.420553] LustreError: 109313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568909 with bad export cookie 16040809467758683056 [ 3377.423594] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3377.431209] LustreError: 109313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3377.659627] Lustre: server umount lustre-MDT0001 complete [ 3384.295352] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 17:08:34 (1786568914) [ 3392.105191] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 3394.259179] Key type lgssc unregistered [ 3394.449445] LNet: 113780:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3394.453820] LNetError: 113780:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3394.471202] LNet: Removed LNI 192.168.201.135@tcp [ 3394.972972] Key type .llcrypt unregistered [ 3394.977407] Key type ._llcrypt unregistered [ 3407.955895] Key type ._llcrypt registered [ 3407.958317] Key type .llcrypt registered [ 3408.054554] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3420.274852] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3420.956279] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3421.015706] alg: No test for adler32 (adler32-zlib) [ 3421.943743] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 3422.110120] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 3423.763242] Key type lgssc registered [ 3424.469805] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3458.197120] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3458.259689] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3459.502258] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3459.525948] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3459.587777] Lustre: lustre-MDT0000: new disk, initializing [ 3459.660530] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3459.685671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3463.825568] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3472.095562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3472.312822] Lustre: lustre-OST0000: new disk, initializing [ 3472.315839] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3472.318637] Lustre: Skipped 1 previous similar message [ 3472.321406] Lustre: 117932:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3472.374460] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3473.562168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3473.572663] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3473.594931] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3477.343630] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3485.519367] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3485.690760] Lustre: lustre-OST0001: new disk, initializing [ 3485.701847] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3485.709378] Lustre: 118794:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3485.785869] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3490.958494] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3493.450717] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3493.459882] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3493.489104] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3498.466241] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3498.475938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3503.586399] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3503.591569] Lustre: Skipped 1 previous similar message [ 3504.511284] Lustre: server umount lustre-MDT0000 complete [ 3509.505270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3512.984403] LustreError: 117120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569045 with bad export cookie 8882213641744414544 [ 3512.992561] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3513.061834] Lustre: server umount lustre-OST0000 complete [ 3515.828787] Lustre: server umount lustre-OST0001 complete [ 3523.693239] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 17:10:54 (1786569054) [ 3542.541041] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 17:11:13 (1786569073) [ 3549.605388] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3567.665654] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:11:38 (1786569098) [ 3598.691069] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3605.601113] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3630.427077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3630.648195] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3630.672422] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3630.726685] Lustre: lustre-MDT0000: new disk, initializing [ 3630.777479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3630.791715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3633.830910] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3641.899152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3641.986174] Lustre: 126545:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3642.020659] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3642.023236] Lustre: Skipped 1 previous similar message [ 3642.076251] Lustre: lustre-MDT0001: new disk, initializing [ 3642.166374] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3642.208324] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3642.219991] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3645.832808] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3653.242942] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3657.508416] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3662.509203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3662.738893] Lustre: lustre-OST0000: new disk, initializing [ 3662.742909] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3662.747614] Lustre: 128510:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3662.832785] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3667.565113] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3671.078390] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3671.088312] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3671.170506] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3675.394823] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3679.032237] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3679.178948] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3682.548653] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3682.738619] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3686.375833] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3686.378649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3686.390961] Lustre: Skipped 2 previous similar messages [ 3690.374642] Lustre: server umount lustre-OST0000 complete [ 3693.538808] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3693.546606] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3693.556688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3693.560344] Lustre: Skipped 1 previous similar message [ 3696.609800] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3701.737348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3701.739963] Lustre: Skipped 3 previous similar messages [ 3707.360458] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3707.608572] Lustre: server umount lustre-MDT0000 complete [ 3710.612566] LustreError: 126539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569242 with bad export cookie 8882213641744415769 [ 3710.619472] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3710.621048] LustreError: 126539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3710.903310] Lustre: server umount lustre-MDT0001 complete [ 3717.470315] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:14:08 (1786569248) [ 3723.101628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3723.490406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3727.109067] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3728.873586] LustreError: 130921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3728.900085] LustreError: 130921:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3732.968972] LustreError: 130922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3734.472312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3738.157509] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3744.163562] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3748.624778] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3753.450318] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3757.929538] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3763.715893] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3772.621414] Lustre: *** cfs_fail_loc=159, val=0*** [ 3788.866802] Lustre: lustre-MDT0000: Client 28cfe2cf-094a-49a4-a95c-668ba174d965 (at 192.168.201.35@tcp) reconnecting [ 3790.376792] Lustre: *** cfs_fail_loc=159, val=0*** [ 3790.382777] Lustre: Skipped 5 previous similar messages [ 3806.761927] Lustre: lustre-MDT0000: Client 28cfe2cf-094a-49a4-a95c-668ba174d965 (at 192.168.201.35@tcp) reconnecting [ 3811.311177] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3811.316423] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3811.326065] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3811.329489] Lustre: Skipped 2 previous similar messages [ 3815.674850] Lustre: server umount lustre-OST0000 complete [ 3819.490745] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3819.496574] Lustre: Skipped 1 previous similar message [ 3824.753825] Lustre: server umount lustre-MDT0000 complete [ 3827.572821] LustreError: 133637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569359 with bad export cookie 8882213641744416644 [ 3827.576170] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3827.578924] LustreError: 133637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3827.859821] Lustre: server umount lustre-MDT0001 complete [ 3834.964236] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 3836.876753] Key type lgssc unregistered [ 3837.033299] LNet: 134694:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3837.039762] LNetError: 134694:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3838.056128] LNet: Removed LNI 192.168.201.135@tcp [ 3838.677337] Key type .llcrypt unregistered [ 3838.680623] Key type ._llcrypt unregistered [ 3853.676456] Key type ._llcrypt registered [ 3853.677960] Key type .llcrypt registered [ 3853.727500] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3854.353546] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3854.365815] alg: No test for adler32 (adler32-zlib) [ 3855.333700] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 3855.553540] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 3857.192211] Key type lgssc registered [ 3857.933808] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3863.506834] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3863.526833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3864.762095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3867.615569] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3872.900391] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3873.125906] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3875.783857] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3880.703713] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3884.489910] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3889.118915] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3889.348997] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3890.430932] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3893.183714] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3897.711859] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3906.003187] Lustre: *** cfs_fail_loc=159, val=0*** [ 3921.456934] Lustre: lustre-MDT0000: Client dea65754-8d56-4be4-9b67-8f2eae5c01d4 (at 192.168.201.35@tcp) reconnecting [ 3923.016468] Lustre: *** cfs_fail_loc=159, val=0*** [ 3938.369852] Lustre: lustre-MDT0001: Client dea65754-8d56-4be4-9b67-8f2eae5c01d4 (at 192.168.201.35@tcp) reconnecting [ 3939.420614] Lustre: *** cfs_fail_loc=159, val=0*** [ 3939.430286] Lustre: Skipped 3 previous similar messages [ 3955.773754] Lustre: lustre-MDT0001: Client dea65754-8d56-4be4-9b67-8f2eae5c01d4 (at 192.168.201.35@tcp) reconnecting [ 3958.489828] Lustre: 139128:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3960.365265] Lustre: *** cfs_fail_loc=159, val=0*** [ 3960.366786] Lustre: Skipped 3 previous similar messages [ 3976.758087] Lustre: lustre-MDT0000: Client dd411fce-1ea1-4abc-b29f-c7d55ee67b6c (at 192.168.201.35@tcp) reconnecting [ 3977.922395] Lustre: *** cfs_fail_loc=159, val=0*** [ 3977.925589] Lustre: Skipped 14 previous similar messages [ 3994.153275] Lustre: lustre-MDT0000: Client dd411fce-1ea1-4abc-b29f-c7d55ee67b6c (at 192.168.201.35@tcp) reconnecting [ 3995.158197] Lustre: 139418:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3998.177424] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3998.181722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3998.189335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4000.297514] Lustre: server umount lustre-OST0000 complete [ 4006.370382] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4006.376987] Lustre: Skipped 1 previous similar message [ 4006.380533] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4006.383477] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4006.383850] Lustre: Skipped 1 previous similar message [ 4008.717665] Lustre: server umount lustre-MDT0000 complete [ 4011.235563] LustreError: 136070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569543 with bad export cookie 7361278990838652900 [ 4011.237841] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4011.241259] LustreError: 136070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4020.825944] Lustre: server umount lustre-MDT0001 complete [ 4027.735252] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4029.527232] Key type lgssc unregistered [ 4029.742606] LNet: 140470:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4029.746300] LNetError: 140470:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4029.757694] LNet: Removed LNI 192.168.201.135@tcp [ 4030.253559] Key type .llcrypt unregistered [ 4030.255695] Key type ._llcrypt unregistered [ 4045.697615] Key type ._llcrypt registered [ 4045.698839] Key type .llcrypt registered [ 4045.766303] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4046.315206] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4046.378481] alg: No test for adler32 (adler32-zlib) [ 4047.297305] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 4047.477187] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4049.080226] Key type lgssc registered [ 4049.753155] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4054.168317] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4054.184700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4055.460138] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4058.476212] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4064.675110] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4064.740241] LustreError: 141862:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4064.755254] LustreError: 141862:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4064.909923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4067.521597] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4072.421095] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4076.395887] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4080.728566] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4080.992261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4084.839304] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4089.508893] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4090.230825] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4100.466107] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4104.883216] Lustre: 144464:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4106.705050] Lustre: 144512:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4107.849883] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4108.475587] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4111.077672] Lustre: 144660:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4111.773578] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4112.436867] Lustre: 144758:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4115.937849] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4115.938136] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4115.942909] LustreError: Skipped 1 previous similar message [ 4115.945081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4115.953755] Lustre: Skipped 1 previous similar message [ 4115.976110] Lustre: Skipped 1 previous similar message [ 4121.058151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4121.411101] Lustre: server umount lustre-OST0000 complete [ 4126.179515] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4126.186657] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4126.192409] Lustre: Skipped 1 previous similar message [ 4126.197783] Lustre: Skipped 1 previous similar message [ 4129.978850] Lustre: server umount lustre-MDT0000 complete [ 4131.299578] LustreError: 141857:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4131.314879] LustreError: 141857:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4132.597450] LustreError: 141843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569664 with bad export cookie 4288635358199574741 [ 4132.599614] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4132.605510] LustreError: 141843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4132.929607] Lustre: server umount lustre-MDT0001 complete [ 4141.255941] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4143.447992] Key type lgssc unregistered [ 4143.704395] LNet: 145818:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4143.710969] LNetError: 145818:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4143.724223] LNet: Removed LNI 192.168.201.135@tcp [ 4144.275537] Key type .llcrypt unregistered [ 4144.276839] Key type ._llcrypt unregistered [ 4161.555597] Key type ._llcrypt registered [ 4161.557390] Key type .llcrypt registered [ 4161.642830] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4162.236019] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4162.334973] alg: No test for adler32 (adler32-zlib) [ 4163.374100] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 4163.581491] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4165.232429] Key type lgssc registered [ 4166.154874] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4172.075123] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4172.095127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4173.437921] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4176.202516] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4182.914231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4183.014748] LustreError: 147211:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4183.026035] LustreError: 147211:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4183.282318] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4186.341301] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4191.470791] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4195.497895] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4200.031223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4200.262722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4204.707838] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4208.429138] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4210.438402] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4217.654790] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4226.148798] Lustre: *** cfs_fail_loc=159, val=0*** [ 4242.479068] Lustre: lustre-MDT0000: Client 38000ab8-7d2f-4b25-a29c-90bd0bafc4d3 (at 192.168.201.35@tcp) reconnecting [ 4248.044695] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4248.049912] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4248.058361] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4248.553896] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4248.570409] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4251.331746] Lustre: server umount lustre-OST0000 complete [ 4255.207322] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4255.216080] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4255.228530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4258.787882] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4258.790720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4258.808580] Lustre: Skipped 1 previous similar message [ 4260.189824] Lustre: server umount lustre-MDT0000 complete [ 4262.986687] LustreError: 147192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569795 with bad export cookie 13345231360378409186 [ 4262.990787] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4262.995313] LustreError: 147192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4263.235950] Lustre: server umount lustre-MDT0001 complete [ 4271.335697] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4273.586230] Key type lgssc unregistered [ 4273.848632] LNet: 150905:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4273.855727] LNetError: 150905:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4273.869838] LNet: Removed LNI 192.168.201.135@tcp [ 4274.523166] Key type .llcrypt unregistered [ 4274.524590] Key type ._llcrypt unregistered [ 4290.407622] Key type ._llcrypt registered [ 4290.411190] Key type .llcrypt registered [ 4290.483403] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4291.001418] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4291.018612] alg: No test for adler32 (adler32-zlib) [ 4291.957372] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 4292.115583] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4293.744176] Key type lgssc registered [ 4294.462290] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4299.945210] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4299.972625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4301.262451] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4304.327315] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4310.425550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4310.708160] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4313.749400] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4319.135638] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4322.918239] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4327.016317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4327.275183] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4330.874372] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4336.147518] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4336.496551] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4346.739328] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4362.485271] Lustre: lustre-MDT0001: evicting 34a672ea-408a-4bbe-aaf0-4db3b031dc35 (at 192.168.201.35@tcp) by administrative request [ 4362.488306] Lustre: Skipped 1 previous similar message [ 4387.813290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4387.816690] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4387.817732] Lustre: Skipped 1 previous similar message [ 4387.817980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4387.826460] LustreError: Skipped 1 previous similar message [ 4392.928760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4392.930638] Lustre: Skipped 2 previous similar messages [ 4393.570952] Lustre: server umount lustre-OST0000 complete [ 4398.050694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4398.050795] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4398.057524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4398.066328] Lustre: Skipped 1 previous similar message [ 4401.546370] Lustre: server umount lustre-MDT0000 complete [ 4403.178092] LustreError: 154046:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4403.193412] LustreError: 154046:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4403.581110] LustreError: 152278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569935 with bad export cookie 6881826432072991438 [ 4403.581807] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4403.588357] LustreError: 152278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4403.748203] Lustre: server umount lustre-MDT0001 complete [ 4409.688315] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4411.535248] Key type lgssc unregistered [ 4411.737143] LNet: 156385:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4411.743929] LNetError: 156385:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4411.755450] LNet: Removed LNI 192.168.201.135@tcp [ 4412.319698] Key type .llcrypt unregistered [ 4412.321811] Key type ._llcrypt unregistered [ 4451.728850] Key type ._llcrypt registered [ 4451.730231] Key type .llcrypt registered [ 4451.779725] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4452.132179] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4452.190599] alg: No test for adler32 (adler32-zlib) [ 4453.073843] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 4453.178269] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4454.776162] Key type lgssc registered [ 4455.290139] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4459.417934] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4459.428121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4460.702674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4462.963566] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4467.560608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4467.764281] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4469.463772] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4472.933360] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4475.521601] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4478.391601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4478.604256] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4481.245960] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4484.308908] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4485.798767] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4492.975974] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4498.402882] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4498.405654] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4498.408630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4499.806884] Lustre: server umount lustre-OST0000 complete [ 4501.847980] Lustre: server umount lustre-MDT0000 complete [ 4503.403741] LustreError: 158827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570035 with bad export cookie 10177084801700746883 [ 4503.408296] LustreError: 158827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4503.408352] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4503.555103] Lustre: server umount lustre-MDT0001 complete [ 4508.465171] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4509.723154] Key type lgssc unregistered [ 4509.885168] LNet: 162404:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4509.894693] LNetError: 162404:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4509.906545] LNet: Removed LNI 192.168.201.135@tcp [ 4510.316845] Key type .llcrypt unregistered [ 4510.317940] Key type ._llcrypt unregistered [ 4523.150647] Key type ._llcrypt registered [ 4523.152110] Key type .llcrypt registered [ 4523.206146] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4523.604559] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4523.652489] alg: No test for adler32 (adler32-zlib) [ 4524.533873] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 4524.646699] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4526.240279] Key type lgssc registered [ 4526.782904] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4534.942591] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 17:27:45 (1786570065) [ 4543.886237] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 4549.083469] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4566.695128] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4566.715837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4567.852732] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4567.867632] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4567.938845] Lustre: lustre-MDT0000: new disk, initializing [ 4567.982792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4567.995276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4570.231696] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4576.272750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4576.350967] Lustre: 166638:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4576.387601] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4576.391289] Lustre: Skipped 1 previous similar message [ 4576.439610] Lustre: lustre-MDT0001: new disk, initializing [ 4576.508196] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4576.535179] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4576.543467] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4579.007386] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4584.388532] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4587.020670] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4590.190708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4590.359407] Lustre: lustre-OST0000: new disk, initializing [ 4590.361538] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4590.365157] Lustre: 168603:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4590.436194] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4593.395277] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4598.363474] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4599.315808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4599.322290] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4599.358691] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4609.508298] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4609.511581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4609.513836] Lustre: Skipped 1 previous similar message [ 4609.518632] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4614.330437] Lustre: server umount lustre-OST0000 complete [ 4617.703459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4617.706237] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4617.711615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4617.716681] Lustre: Skipped 1 previous similar message [ 4619.745238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4619.752864] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4622.569404] Lustre: server umount lustre-MDT0000 complete [ 4624.569570] LustreError: 166632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570156 with bad export cookie 17652138469426148800 [ 4624.574397] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4624.578304] LustreError: 166632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4624.786622] Lustre: server umount lustre-MDT0001 complete [ 4630.292167] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4631.826170] Key type lgssc unregistered [ 4632.014860] LNet: 170512:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4632.019351] LNetError: 170512:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4633.063573] LNet: Removed LNI 192.168.201.135@tcp [ 4633.528489] Key type .llcrypt unregistered [ 4633.529732] Key type ._llcrypt unregistered [ 4645.662567] Key type ._llcrypt registered [ 4645.664652] Key type .llcrypt registered [ 4645.708310] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4646.205164] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4646.223464] alg: No test for adler32 (adler32-zlib) [ 4647.125015] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 4647.244636] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4648.856147] Key type lgssc registered [ 4649.436773] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4653.190428] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4653.211116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4654.423883] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4656.743167] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4661.296145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4661.507087] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4663.538296] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4667.299743] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4670.175416] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4673.677329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4673.849261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4676.759810] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4678.952512] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4680.286180] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4689.396451] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4689.409100] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4689.414083] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4689.418248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.424841] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4689.908124] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4689.924685] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4689.932448] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4689.939717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.949301] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4690.487323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.057877] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4691.061433] Lustre: Skipped 1 previous similar message [ 4691.077653] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4691.087613] Lustre: Skipped 1 previous similar message [ 4691.093760] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4691.095960] Lustre: Skipped 1 previous similar message [ 4691.099824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.107922] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4691.112210] Lustre: Skipped 1 previous similar message [ 4691.673387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.121595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.552578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.095897] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4693.101244] Lustre: Skipped 3 previous similar messages [ 4693.115763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4693.125933] Lustre: Skipped 3 previous similar messages [ 4693.128723] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4693.132608] Lustre: Skipped 3 previous similar messages [ 4693.134979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.141812] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4693.148187] Lustre: Skipped 3 previous similar messages [ 4693.670507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.204235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.754690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.186958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.624607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.069461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.583345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.051811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.536580] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4697.539867] Lustre: Skipped 8 previous similar messages [ 4697.547191] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4697.551533] Lustre: Skipped 8 previous similar messages [ 4697.553751] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4697.556508] Lustre: Skipped 8 previous similar messages [ 4697.558591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.571361] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4697.577379] Lustre: Skipped 8 previous similar messages [ 4698.108620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.563388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.038384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.448153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.889754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.314754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.771272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.225342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.661402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.150739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.730850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.346173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.867745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.458918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.983913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.481076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.019581] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4706.023563] Lustre: Skipped 16 previous similar messages [ 4706.038725] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4706.048965] Lustre: Skipped 16 previous similar messages [ 4706.053583] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4706.056916] Lustre: Skipped 16 previous similar messages [ 4706.059503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.064298] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4706.071199] Lustre: Skipped 16 previous similar messages [ 4706.546976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.123748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.592513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.108554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.648821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.079043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.517928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.008501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.431958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.883080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.367554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.812554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.252320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.688994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.126474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.645667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.165320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.585995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.978468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.501669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.967631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.484742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.973422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.392965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.900640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.425188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.023024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.512525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.965505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.469879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.056251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.562223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.069490] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4722.072090] Lustre: Skipped 32 previous similar messages [ 4722.082687] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4722.088391] Lustre: Skipped 32 previous similar messages [ 4722.091676] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4722.094548] Lustre: Skipped 32 previous similar messages [ 4722.096968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.104812] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4722.110986] Lustre: Skipped 32 previous similar messages [ 4722.612394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.239063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.806894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.278930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.811374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.339822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.760705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.191234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.695604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.153737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.616654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.097550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.596079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.122953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.638845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.088908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.539240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.928287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.338899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.894033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.329903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.781932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.220622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.658876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.043913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.478069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.863659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.341842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.868201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.304952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.738643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.118200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.494256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.904964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.402123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.824892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.308977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.830981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.307379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.803823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.323019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.700905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.083458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.524878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.884094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.299651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.665105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.070536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.498235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.915718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.336903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.711985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.094970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.495792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.867201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.254476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.647556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.006570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.350538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.717265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.122102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.495986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.871724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.229316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.568366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.923988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.271680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.635083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.056667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.494299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.902383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.294514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.696665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.199925] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4754.202145] Lustre: Skipped 73 previous similar messages [ 4754.209397] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4754.212835] Lustre: Skipped 73 previous similar messages [ 4754.214394] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4754.216312] Lustre: Skipped 73 previous similar messages [ 4754.217827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.220885] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4754.223521] Lustre: Skipped 73 previous similar messages [ 4754.694457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.125102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.520174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.973485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.370736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.858634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.245398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.651203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.045280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.480134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.897295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.367353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.815853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.273634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.699629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.123111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.494605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.860174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.219361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.635815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.085695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.556255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.977667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.553847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.065348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.471936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.877132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.243199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.609852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.007575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.419358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.804772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.181497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.571749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.980313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.435090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.872875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.303463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.733969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.128518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.498942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.883211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.230048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.621367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.981101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.362413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.761988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.138921] LustreError: 171926:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4774.144324] LustreError: 171926:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4774.145399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.513740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.913090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.324054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.703567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.082300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.513943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.933913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.329279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.745710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.198368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.647167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.079560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.502460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.993593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.524145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.055810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.553261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.054373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.444253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.839648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.226394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.639900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.047854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.484656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.858403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.325575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.813403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.303633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.841959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.235885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.650851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.095068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.517638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.066754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.530122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.003822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.494089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.970386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.516623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.966264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.396904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.800612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.246783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.731040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.178253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.630086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.078988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.488162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.854471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.229616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.595322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.945177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.301588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.672339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.083077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.516413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.942361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.304193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.657337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.007387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.353146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.714827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.088230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.446208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.841379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.241847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.633906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.984307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.331627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.672238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.022501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.387660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.757954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.158897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.504345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.847200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.280889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.678611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.016730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.376965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.763801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.174394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.557192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.988793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.387538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.798110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.226251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.580379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.963508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.340283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.703199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.113723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.484788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.909670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.304119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.791630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.282610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.694555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.088563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.489069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.880170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.265421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.692661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.134883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.566823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.020652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.438237] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4818.441744] Lustre: Skipped 154 previous similar messages [ 4818.447499] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4818.452087] Lustre: Skipped 154 previous similar messages [ 4818.454124] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4818.456673] Lustre: Skipped 154 previous similar messages [ 4818.459576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.463966] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4818.466282] Lustre: Skipped 154 previous similar messages [ 4818.869523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.251864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.629232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.023470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.428400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.887424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.305445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.747554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.117278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.502896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.971599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.390603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.794209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.260863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.738658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.220430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.684331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.096624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.468662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.873296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.372622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.790532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.266763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.707837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.128657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.601232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.131375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.633373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.169425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.683621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.097354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.497122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.936215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.341691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.794360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.267953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.686660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.183729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.663792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.138747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.650531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.096490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.529306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.019736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.480530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.964232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.426696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.815496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.265526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.685076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.080209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.534144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.002253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.495425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.058062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.548877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.971275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.394542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.850344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.276919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.705180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.153690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.586743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.044187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.566929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.045656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.533676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.999848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.447219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.925575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.421931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.934501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.377333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.823448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.323148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.734497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.199563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.683798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.213928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.695313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.131474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.579866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.035395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.486243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.904000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.346215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.814714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.259953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.681077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.089587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.498190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.962603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.336971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.760180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.159153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.590382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.966484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.348582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.776996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.190876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.651820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.077170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.488445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.904427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.413218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.858383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.391499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.950296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.400176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.810214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.208735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.718468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.207760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.713084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.198572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.639423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.128236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.595728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.046049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.427670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.776798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.191481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.605798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.978535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.371973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.805275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.171742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.522705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.906152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.291879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.694652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.096236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.556399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.903078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.281666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.641203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.011177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.377297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.754251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.150312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.490952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.832526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.163175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.491897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.892340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.250555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.622712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.969332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.326685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.693648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.050218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.396163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.756194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.119156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.500089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.871380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.365310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.798197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.197347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.566795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.913021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.278278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.629915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.970924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.356592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.740061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.123964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.490845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.870361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.250906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.596353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.954100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.330658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.679084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.038649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.389834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.747134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.157685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.603358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.065931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.476211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.856991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.215241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.579578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.957891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.303840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.649442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.004440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.345158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.678714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.037723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.393804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.745823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.076497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.417925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.771956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.103571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.446054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.770628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.097184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.432428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.788212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.142428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.508470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.962102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.438456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.809433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.179854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.534718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.894948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.286617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.643847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.039797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.408858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.782603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.128225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.485031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.815956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.165788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.542117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.876392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.207492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.540058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.894041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.284637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.692981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.044031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.406474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.775667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.239050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.642720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.100426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.538994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.952735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.362156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.853519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.297246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.759783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.269735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.700207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.203208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.723146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.138700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.568102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.122351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.572630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.997628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.460663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.898352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.361054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.880180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.318920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.704874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.113208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.552200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.960951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.366444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.795507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.171312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.578121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.070076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.492245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.885419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.295316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.725509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.156264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.569879] LustreError: 171926:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4929.573720] LustreError: 171926:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4929.575322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.959039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.307718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.660681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.002209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.378455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.771721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.157327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.525104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.904380] LustreError: 171926:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4932.908656] LustreError: 171926:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4932.914398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.300865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.685214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.065582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.450966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.872062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.329274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.805394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.237183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.676925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.097128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.487975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.944635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.414401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.860992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.289533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.697021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.085413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.499790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.927710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.363615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.736663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.170086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.566746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.990786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.376480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.826936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.260677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.669902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.078052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.488729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.918224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.364381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.912369] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4946.915548] Lustre: Skipped 308 previous similar messages [ 4946.929863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4946.935107] Lustre: Skipped 308 previous similar messages [ 4946.936910] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4946.940105] Lustre: Skipped 308 previous similar messages [ 4946.941920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.947446] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4946.950986] Lustre: Skipped 308 previous similar messages [ 4947.378616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.768164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.176722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.530608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.894426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.296830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.661945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.030781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.432271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.811274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.221974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.607162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.991206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.445389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.888323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.306201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.770046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.204287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.624361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.061568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.497974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.968339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.395037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.805233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.250457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.713345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.192534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.686842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.125880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.558375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.935161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.363944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.884368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.320140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.769119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.235984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.666711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.091184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.486325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.987799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.452041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.872684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.321977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.815665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.312715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.746258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.160972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.634303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.055339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.531738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.028482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.509765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.049829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.535446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.978721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.420099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.835479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.186798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.542307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.899160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.286560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.690099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.112643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.675325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.167393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.633972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.198200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.754663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.212070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.767095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.242170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.684822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.140429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.551884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.007520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.395360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.738897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.113125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.468008] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.832073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.210184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.568723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.934443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.280686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.628566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.005697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.362040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.748322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.161513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.633186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.024269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.394194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.769142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.115849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.460086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.816774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.168941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.528156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.899330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.257473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.591150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.952621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.312518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.659041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.005895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.362382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.726750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.100377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.454931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.794105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.131093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.491657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.892815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.257708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.595570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.991764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.367403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.741986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.114086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.475667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.852946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.212789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.596228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.956250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.304723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.649262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.989721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.355391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.713498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.060189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.414352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.757646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.151747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.543913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.934816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.298669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.667755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.034552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.412258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.768408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.118691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.494229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.849138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.245927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.623328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.000426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.375519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.746107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.137947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.513590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.901341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.292968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.692659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.094076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.522920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.997767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.530447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.000371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.405924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.825268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.249663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.617113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.004915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.377384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.733236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.128670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.511726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.930376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.356819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.840527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.245429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.674455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.057963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.441396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.821302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.183919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.523537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.866777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.215679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.561878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.897400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.268379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.605387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.950945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.292845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.645581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.004199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.355082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.744316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.129211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.513882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.948595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.334270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.756231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.146445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.525769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.875160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.259967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.633652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.048176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.468038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.875067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.302546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.718603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.126992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.508273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.892596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.260475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.642343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.032458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.420492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.841764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.233803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.630884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.022834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.446297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.820947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.222893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.683414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.143200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.657341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.110820] LustreError: 171926:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5036.116043] LustreError: 171926:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5036.120110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.588395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.100649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.640662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.133228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.654560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.160673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.597556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.070250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.539897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.993409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.419053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.840250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.310348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.802809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.219300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.680827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.140385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.624548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.169357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.644496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.074514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.531737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.004718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.524494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.990885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.475713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.954763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.460481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.940914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.379227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.800569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.236066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.652528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.162756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.642385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.112777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.587090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.988779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.406810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.795448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.201263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.611148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.992261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.356181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.714320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.129586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.586564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.020792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.485888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.927582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.363331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.771702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.193437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.636122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.089685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.475747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.907697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.313572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.808385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.199098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.596925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.950437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.310019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.716183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.141437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.510846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.905473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.318626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.764389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.199066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.641679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.112752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.584934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.051167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.483302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.900821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.295754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.666237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.048294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.404586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.787790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.162939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.538311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.931524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.301731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.668086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.037796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.386866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.762022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.142743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.541081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.951439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.340529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.772804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.204393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.636948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.065406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.484101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.883563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.269937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.632601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.023353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.444491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.899903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.266324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.660400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.095356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.508358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.922868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.374813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.771491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.182549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.585358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.011040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.452951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.896435] LustreError: 171926:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5085.900199] LustreError: 171926:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5085.901810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.284608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.677204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.048979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.436253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.798241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.164835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.592910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.965340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.343206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.709080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.064661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.434725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.798338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.162736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.573568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.994842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.424665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.867064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.261644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.644749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.998600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.342792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.712506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.047581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.408785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.738669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.076428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.425118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.763362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.112109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.462307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.796740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.148026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.477250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.827272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.161305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.513144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.879618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.234318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.572809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.956636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.293444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.668911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.032180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.398324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.770250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.144581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.531447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.932674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.312871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.683162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.063915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.436693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.795048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.159219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.496097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.845022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.028583] LustreError: 173723:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5241.031475] LustreError: 171926:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5326.307883] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5326.308175] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5326.313547] Lustre: Skipped 395 previous similar messages [ 5326.317365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5326.321272] Lustre: Skipped 1 previous similar message [ 5331.427196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5331.432398] Lustre: Skipped 1 previous similar message [ 5331.515108] Lustre: server umount lustre-OST0000 complete [ 5336.546398] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5336.549621] Lustre: Skipped 1 previous similar message [ 5339.294162] Lustre: server umount lustre-MDT0000 complete [ 5340.708356] LustreError: 171888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570873 with bad export cookie 5396650014988056052 [ 5340.710363] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5340.716551] LustreError: 171888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5340.845168] Lustre: server umount lustre-MDT0001 complete [ 5345.042649] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 5346.125643] Key type lgssc unregistered [ 5346.264397] LNet: 226946:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5346.267953] LNetError: 226946:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5346.277343] LNet: Removed LNI 192.168.201.135@tcp [ 5346.624165] Key type .llcrypt unregistered [ 5346.625178] Key type ._llcrypt unregistered [ 5356.637880] Key type ._llcrypt registered [ 5356.639347] Key type .llcrypt registered [ 5356.682532] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5357.139756] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5357.151236] alg: No test for adler32 (adler32-zlib) [ 5358.006672] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 5358.096105] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 5359.680138] Key type lgssc registered [ 5360.087093] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5362.777104] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5362.783450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5363.939884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5365.373111] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5368.130373] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5368.254889] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5369.596561] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5372.273197] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5374.343593] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5376.745172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5376.878763] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5378.884332] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5381.580444] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5389.160767] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 5394.830596] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5395.492690] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5395.497847] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5395.500777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.504915] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5395.504939] LustreError: 228356:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5398.030590] Lustre: lustre-OST0000: Not available for connect from 192.168.201.35@tcp (stopping) [ 5398.496970] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5398.499714] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5399.521433] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5399.521864] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5399.523612] Lustre: Skipped 1 previous similar message [ 5402.333584] Lustre: server umount lustre-OST0000 complete [ 5415.584150] LustreError: 228356:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5430.909565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5431.048232] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5433.309074] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5436.298989] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5439.941469] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 3 sec [ 5440.293239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.297160] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5441.984648] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8af2e0ca3800.ost_server_uuid 50 [ 5442.591233] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8af2e0ca3800.ost_server_uuid in FULL state after 0 sec [ 5445.601772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5445.610157] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5449.697636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5449.937613] Lustre: server umount lustre-OST0000 complete [ 5454.818392] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5454.819223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5454.822472] Lustre: Skipped 1 previous similar message [ 5454.825194] Lustre: Skipped 1 previous similar message [ 5457.554335] Lustre: server umount lustre-MDT0000 complete [ 5458.855043] LustreError: 230178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570991 with bad export cookie 13614936254639879951 [ 5458.857888] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5458.859480] LustreError: 230178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5458.976790] Lustre: server umount lustre-MDT0001 complete [ 5462.857231] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 5463.846651] Key type lgssc unregistered [ 5463.990362] LNet: 233563:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5463.992562] LNetError: 233563:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5464.000314] LNet: Removed LNI 192.168.201.135@tcp [ 5464.310897] Key type .llcrypt unregistered [ 5464.311917] Key type ._llcrypt unregistered [ 5476.169365] Key type ._llcrypt registered [ 5476.170386] Key type .llcrypt registered [ 5476.212299] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5476.538944] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5476.568926] alg: No test for adler32 (adler32-zlib) [ 5477.426171] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 5477.526776] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 5479.112157] Key type lgssc registered [ 5479.459758] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5482.647448] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5482.653421] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5483.811454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5485.258529] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5488.242609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5488.374080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5489.820790] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5490.849452] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5493.112734] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5493.233618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5494.248904] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 5495.255527] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5498.150369] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5498.220164] Lustre: lustre-OST0001: new disk, initializing [ 5498.222363] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5498.225403] Lustre: 237577:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5498.258700] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5500.262162] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5501.131751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5501.135643] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5501.168354] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5505.933865] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5508.708647] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5516.770880] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5516.772606] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5516.774919] Lustre: Skipped 2 previous similar messages [ 5516.778560] Lustre: Skipped 3 previous similar messages [ 5521.889269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5521.891737] Lustre: Skipped 2 previous similar messages [ 5522.148659] Lustre: server umount lustre-MDT0000 complete [ 5523.393148] LustreError: 235247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571055 with bad export cookie 18411206337219636371 [ 5523.396625] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5523.397564] LustreError: 235247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5523.514490] Lustre: server umount lustre-MDT0001 complete [ 5535.237244] Lustre: server umount lustre-OST0000 complete [ 5545.707684] Lustre: server umount lustre-OST0001 complete [ 5553.148464] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5556.582524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5556.730962] LustreError: 240676:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5556.738990] LustreError: 240676:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5556.749336] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5558.102436] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5560.935685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5562.486507] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5563.516651] Lustre: 241819:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5565.751026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5565.867759] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5565.869736] Lustre: Skipped 1 previous similar message [ 5566.883138] LustreError: 242170:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5566.887040] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 5567.922138] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5570.981123] LustreError: 242171:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5570.990698] LustreError: 242171:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5571.218415] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5573.388820] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5576.613688] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5577.894173] Lustre: 243686:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5586.916298] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5586.918483] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5586.921282] Lustre: Skipped 4 previous similar messages [ 5586.921583] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5586.923540] Lustre: Skipped 4 previous similar messages [ 5588.516834] Lustre: server umount mylustre-MDT0000 complete [ 5589.884807] LustreError: 240708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571122 with bad export cookie 18411206337219638996 [ 5589.886996] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5589.889240] LustreError: 240708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5590.016051] Lustre: server umount mylustre-MDT0001 complete [ 5601.217982] Lustre: server umount mylustre-OST0000 complete [ 5612.631770] Lustre: server umount mylustre-OST0001 complete [ 5619.319505] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5622.665627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5622.790803] LustreError: 245989:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5622.807121] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5622.808663] Lustre: Skipped 1 previous similar message [ 5624.218179] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5627.110572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5628.488121] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5629.415312] Lustre: 247133:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5631.523064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5632.676107] LustreError: 247485:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5632.680232] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 5633.377495] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5635.980544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5637.820360] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5640.901450] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5642.085849] Lustre: 249002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5647.334111] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5647.335558] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5647.337450] Lustre: Skipped 2 previous similar messages [ 5647.339942] Lustre: Skipped 3 previous similar messages [ 5650.021834] Lustre: server umount tfs-MDT0000 complete [ 5651.278050] LustreError: 245966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571183 with bad export cookie 18411206337219640662 [ 5651.280823] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5651.283454] LustreError: 245966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5651.390222] Lustre: server umount tfs-MDT0001 complete [ 5662.852531] Lustre: server umount tfs-OST0000 complete [ 5674.331647] Lustre: server umount tfs-OST0001 complete [ 5680.852211] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5684.063588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5684.188372] LustreError: 251247:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5684.205072] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5684.206876] Lustre: Skipped 3 previous similar messages [ 5685.554891] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5688.222033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5689.672433] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5690.600485] Lustre: 252388:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5692.671773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5693.801190] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 5694.582911] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5697.397378] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5699.329874] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5702.368062] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5703.451712] Lustre: 254256:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5705.438324] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 17:47:16 (1786571236) [ 5719.011163] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5719.013368] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5719.013571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5719.015058] Lustre: Skipped 2 previous similar messages [ 5723.286542] Lustre: server umount lustre-MDT0000 complete [ 5724.131126] LustreError: 251242:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5724.136182] LustreError: 251242:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 5724.448052] LustreError: 251991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571256 with bad export cookie 18411206337219642097 [ 5724.451426] LustreError: 251991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5724.451724] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5729.249752] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5729.250585] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5729.253400] Lustre: Skipped 3 previous similar messages [ 5729.257915] Lustre: Skipped 4 previous similar messages [ 5730.603036] Lustre: server umount lustre-MDT0001 complete [ 5746.144086] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5746.177037] Lustre: server umount lustre-OST0000 complete [ 5753.434968] Lustre: server umount lustre-OST0001 complete [ 5755.440025] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 5758.495875] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5769.496327] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5769.496327] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5769.496327] [ 5769.504386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5769.580848] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5769.591540] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5769.624639] Lustre: lustre-MDT0000: new disk, initializing [ 5769.645582] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5769.647940] Lustre: Skipped 3 previous similar messages [ 5769.653660] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5770.948234] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5774.927800] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 5774.927800] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5774.927800] [ 5774.938215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5774.977305] Lustre: 257442:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5774.999933] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5775.001595] Lustre: Skipped 1 previous similar message [ 5775.033325] Lustre: lustre-MDT0001: new disk, initializing [ 5775.073821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5775.076735] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5776.390948] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5780.411321] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5780.508437] Lustre: lustre-OST0000: new disk, initializing [ 5780.510104] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5780.512865] Lustre: 259025:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5781.559308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5781.563217] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5781.598195] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5782.430222] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5786.464046] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5786.520630] Lustre: lustre-OST0001: new disk, initializing [ 5786.522328] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5786.524718] Lustre: 259898:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5787.710054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5787.712790] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5787.741420] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5788.345126] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5792.395189] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 17:48:43 (1786571323) [ 5795.752574] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 17:48:47 (1786571327) [ 5807.073444] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5807.076423] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5807.080291] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5807.082028] Lustre: Skipped 1 previous similar message [ 5821.408071] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5821.442166] Lustre: server umount lustre-OST0000 complete [ 5823.458533] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5823.458647] LustreError: 260621:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5823.461717] Lustre: Skipped 3 previous similar messages [ 5823.466137] LustreError: 260621:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5828.828074] Lustre: server umount lustre-MDT0000 complete [ 5829.978252] LustreError: 257437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571362 with bad export cookie 18411206337219643427 [ 5829.981508] LustreError: 257437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5829.981678] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5836.154547] Lustre: server umount lustre-MDT0001 complete [ 5839.780672] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 5922.272170] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5922.402235] Lustre: server umount lustre-OST0001 complete [ 5926.212484] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 5941.216588] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5964.630526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5964.781690] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5964.798728] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5964.852945] Lustre: lustre-MDT0000: new disk, initializing [ 5964.903281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5964.907392] Lustre: Skipped 3 previous similar messages [ 5964.917851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5967.013574] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5973.405262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5973.459308] Lustre: 264713:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5973.596067] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5975.724762] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5981.036111] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5984.446143] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5988.736726] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5988.876685] Lustre: lustre-OST0000: new disk, initializing [ 5988.878506] Lustre: Skipped 1 previous similar message [ 5988.881435] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5988.884855] Lustre: Skipped 2 previous similar messages [ 5988.887762] Lustre: 266678:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5990.368159] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5990.371852] Lustre: Skipped 1 previous similar message [ 5990.374225] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5990.456469] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5992.171293] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5997.561664] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6005.733804] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6005.734931] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6005.739191] LustreError: Skipped 2 previous similar messages [ 6005.740123] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6005.748448] Lustre: Skipped 2 previous similar messages [ 6005.750537] Lustre: Skipped 14 previous similar messages [ 6007.735951] Lustre: server umount lustre-OST0000 complete [ 6015.895854] Lustre: server umount lustre-MDT0000 complete [ 6015.970858] LustreError: 264719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6015.977553] LustreError: 264719:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 6017.530924] LustreError: 266676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571549 with bad export cookie 18411206337219644897 [ 6017.531327] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6017.535839] LustreError: 266676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6017.690643] Lustre: server umount lustre-MDT0001 complete [ 6023.346989] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6024.991527] Key type lgssc unregistered [ 6025.124343] LNet: 268438:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6025.128108] LNetError: 268438:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6025.135326] LNet: Removed LNI 192.168.201.135@tcp [ 6025.499136] Key type .llcrypt unregistered [ 6025.500468] Key type ._llcrypt unregistered [ 6039.855657] Key type ._llcrypt registered [ 6039.856916] Key type .llcrypt registered [ 6039.908199] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6040.382568] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6040.393067] alg: No test for adler32 (adler32-zlib) [ 6041.288474] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 6041.471065] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6043.106495] Key type lgssc registered [ 6043.729746] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6048.217132] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6048.226065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6049.433032] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6051.529944] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6056.581047] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6056.804743] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6059.451543] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6064.470631] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6068.047862] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6072.371110] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6072.596739] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6073.658952] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6076.862064] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6081.880611] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6082.530806] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6084.065427] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6084.069143] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6084.075444] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6087.141679] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6087.142274] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6087.142616] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6089.865947] Lustre: server umount lustre-OST0000 complete [ 6092.258246] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6092.260514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6092.266861] Lustre: Skipped 1 previous similar message [ 6092.272990] Lustre: Skipped 2 previous similar messages [ 6097.377552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6097.384570] Lustre: Skipped 1 previous similar message [ 6098.317061] Lustre: server umount lustre-MDT0000 complete [ 6100.530277] LustreError: 269858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571632 with bad export cookie 10313644961067479932 [ 6100.533864] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6100.534937] LustreError: 269858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6100.739669] Lustre: server umount lustre-MDT0001 complete [ 6106.966777] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6108.219751] Key type lgssc unregistered [ 6108.387517] LNet: 273462:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6108.390260] LNetError: 273462:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6108.401565] LNet: Removed LNI 192.168.201.135@tcp [ 6108.804039] Key type .llcrypt unregistered [ 6108.805471] Key type ._llcrypt unregistered [ 6117.043450] Key type ._llcrypt registered [ 6117.045026] Key type .llcrypt registered [ 6117.097893] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6117.635062] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6117.670761] alg: No test for adler32 (adler32-zlib) [ 6118.554214] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 6118.673861] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6120.256141] Key type lgssc registered [ 6120.807022] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6124.822337] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6124.828989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6140.256353] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6145.376320] LustreError: 274632:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.135@tcp: failed processing log, type 4: rc = -110 [ 6172.000203] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6174.096662] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6176.811760] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6177.636874] Lustre: server umount lustre-OST0000 complete [ 6183.705187] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6184.838393] Key type lgssc unregistered [ 6184.969415] LNet: 276262:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6184.972588] LNetError: 276262:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6184.984347] LNet: Removed LNI 192.168.201.135@tcp [ 6185.319364] Key type .llcrypt unregistered [ 6185.321063] Key type ._llcrypt unregistered [ 6200.561979] Key type ._llcrypt registered [ 6200.563143] Key type .llcrypt registered [ 6200.607318] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6200.950333] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6200.986071] alg: No test for adler32 (adler32-zlib) [ 6201.856030] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 6201.957963] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6203.544169] Key type lgssc registered [ 6203.912791] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6208.479719] loop0: detected capacity change from 0 to 204800000 [ 6208.543157] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6208.548654] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6208.676506] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6209.047149] loop0: detected capacity change from 0 to 204800000 [ 6209.105020] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6209.109444] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6209.233208] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6209.610367] loop0: detected capacity change from 0 to 204800000 [ 6209.713770] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6209.717442] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6209.760528] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6210.151495] loop0: detected capacity change from 0 to 204800000 [ 6210.206856] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6210.631837] loop0: detected capacity change from 0 to 204800000 [ 6210.640839] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6212.120234] loop0: detected capacity change from 0 to 204800000 [ 6212.129155] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6213.509414] loop0: detected capacity change from 0 to 204800000 [ 6213.521463] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6214.893150] loop0: detected capacity change from 0 to 204800000 [ 6214.908839] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6216.268735] loop0: detected capacity change from 0 to 204800000 [ 6216.295187] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6216.307983] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6216.340660] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6216.343644] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6216.346685] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6216.350591] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6216.354565] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6216.358508] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6216.362065] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6216.365515] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6216.369726] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6216.373879] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6216.378193] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6216.383640] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6216.387847] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6216.394703] Lustre: 279653:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6219.215621] Lustre: server umount MGS complete [ 6219.543441] loop0: detected capacity change from 0 to 204800000 [ 6219.572934] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6219.728475] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6219.744474] Lustre: lustre-MDT0000: Aborting client recovery [ 6220.072428] loop1: detected capacity change from 0 to 204800000 [ 6220.101498] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6220.141021] Lustre: 280086:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6220.146924] Lustre: 280086:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6220.152690] Lustre: 280086:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6220.157175] Lustre: 280086:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6220.161220] Lustre: 280086:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6220.165315] Lustre: 280086:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6220.170119] Lustre: 280086:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6220.177882] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6220.240612] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6220.269227] Lustre: lustre-MDT0001: Aborting client recovery [ 6220.627533] loop2: detected capacity change from 0 to 204800000 [ 6220.658722] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6220.692158] Lustre: 280169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6220.696374] Lustre: 280169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6220.699645] Lustre: 280169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6220.703467] Lustre: 280169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6220.706629] Lustre: 280169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6220.710608] Lustre: 280169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6220.714732] Lustre: 280169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6220.762412] Lustre: Found index 0 for lustre-OST0000, updating log [ 6220.772842] Lustre: 280186:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6220.795750] Lustre: lustre-OST0000: Aborting recovery [ 6221.122738] loop3: detected capacity change from 0 to 204800000 [ 6221.152594] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6221.184989] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6221.190791] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6221.195865] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6221.200424] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6221.205691] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6221.210451] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6221.214242] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6221.245728] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6221.248079] Lustre: Skipped 1 previous similar message [ 6226.406463] LustreError: 280030:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 6230.501360] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6230.501587] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6230.514277] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6230.514297] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6230.517155] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6230.528660] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6232.053223] LustreError: 279996:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 144us: rc = -2 [ 6245.856488] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6245.856588] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6245.857091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6245.860574] Lustre: Skipped 2 previous similar messages [ 6249.568982] Lustre: server umount lustre-MDT0000 complete [ 6249.912644] LustreError: 279981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571782 with bad export cookie 4621049370965444543 [ 6249.913831] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6249.919745] LustreError: 279981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6250.976873] LustreError: 279993:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6250.976919] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6250.977380] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6250.977384] Lustre: Skipped 3 previous similar messages [ 6250.992259] LustreError: 279993:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6250.995849] Lustre: Skipped 2 previous similar messages [ 6255.008580] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6255.008579] LustreError: 279994:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6255.016355] Lustre: Skipped 2 previous similar messages [ 6260.128659] LustreError: 279995:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6260.128676] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6260.133509] LustreError: 279995:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6260.135902] Lustre: Skipped 1 previous similar message [ 6264.288111] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6264.381494] Lustre: server umount lustre-MDT0001 complete [ 6279.136144] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6279.181847] Lustre: server umount lustre-OST0000 complete [ 6285.650905] Lustre: server umount lustre-OST0001 complete [ 6288.416761] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:56:59 (1786571819) [ 6297.776966] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 6300.900871] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6313.049827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6313.130776] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6313.141130] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6313.177959] Lustre: lustre-MDT0000: new disk, initializing [ 6313.202099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6313.208185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6314.565363] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6318.683856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6318.724166] Lustre: 284321:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6318.749656] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6318.751451] Lustre: Skipped 1 previous similar message [ 6318.788169] Lustre: lustre-MDT0001: new disk, initializing [ 6318.821980] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6318.833698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6318.836301] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6320.120480] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6323.809944] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6325.550604] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6327.752452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6327.862405] Lustre: lustre-OST0000: new disk, initializing [ 6327.863855] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6327.867302] Lustre: 286290:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6327.869520] Lustre: 286290:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 6327.900486] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6328.633548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6328.636443] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6328.664066] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6329.772758] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6333.345937] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6335.619340] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6339.448574] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6349.144317] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6354.652808] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6398.435349] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6398.436752] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6398.440672] Lustre: Skipped 1 previous similar message [ 6398.444103] Lustre: Skipped 1 previous similar message [ 6402.580182] Lustre: server umount lustre-OST0000 complete [ 6406.112989] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6406.115741] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6408.673321] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6408.677690] Lustre: Skipped 2 previous similar messages [ 6418.400115] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6418.477884] Lustre: server umount lustre-MDT0000 complete [ 6419.773365] LustreError: 287440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571952 with bad export cookie 4621049370965447252 [ 6419.775845] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6419.778487] LustreError: 287440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6419.885848] Lustre: server umount lustre-MDT0001 complete [ 6422.817607] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6424.170544] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6425.764268] Lustre: server umount MGS complete [ 6429.870568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6430.031243] LustreError: 290292:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6430.038739] LustreError: 290305:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6430.056041] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6431.508981] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6434.806775] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6436.550519] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6439.345028] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6441.754909] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6444.471938] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6446.671221] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6449.263311] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6480.354452] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6480.356094] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6480.356465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6480.356470] Lustre: Skipped 4 previous similar messages [ 6480.358190] Lustre: Skipped 2 previous similar messages [ 6484.047811] Lustre: server umount lustre-OST0000 complete [ 6491.413922] Lustre: server umount lustre-MDT0000 complete [ 6492.701312] LustreError: 290273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572025 with bad export cookie 4621049370965449093 [ 6492.703747] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6492.705541] LustreError: 290273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6492.822447] Lustre: server umount lustre-MDT0001 complete [ 6496.719800] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6497.753470] Key type lgssc unregistered [ 6497.874375] LNet: 294256:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6497.877562] LNetError: 294256:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6497.888334] LNet: Removed LNI 192.168.201.135@tcp [ 6498.193078] Key type .llcrypt unregistered [ 6498.194438] Key type ._llcrypt unregistered [ 6510.586994] Key type ._llcrypt registered [ 6510.588204] Key type .llcrypt registered [ 6510.628617] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 6517.072657] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6517.524122] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6517.530618] alg: No test for adler32 (adler32-zlib) [ 6518.382022] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 6518.471855] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6520.056112] Key type lgssc registered [ 6520.444146] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6532.624494] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6532.632194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6533.711566] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6533.721968] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6533.758395] Lustre: lustre-MDT0000: new disk, initializing [ 6533.779573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6533.784858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6535.137204] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6539.266812] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6539.310070] Lustre: 298195:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6539.332173] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6539.333478] Lustre: Skipped 1 previous similar message [ 6539.368342] Lustre: lustre-MDT0001: new disk, initializing [ 6539.403902] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6539.416529] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6539.419794] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6540.809635] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6544.692035] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6546.738913] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6549.251902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6549.361834] Lustre: lustre-OST0000: new disk, initializing [ 6549.363685] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6549.366767] Lustre: 300162:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6549.402376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6551.563319] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6555.488586] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6556.159392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6556.163492] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6556.199520] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6564.173385] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6572.058357] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6580.726726] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6587.276865] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6622.689829] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6622.691321] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6622.692078] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6622.692324] LustreError: Skipped 1 previous similar message [ 6622.695522] Lustre: Skipped 1 previous similar message [ 6627.404707] Lustre: server umount lustre-OST0000 complete [ 6631.904748] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6631.906770] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6631.910086] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6631.912103] Lustre: Skipped 1 previous similar message [ 6632.929108] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6632.929914] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6634.723450] Lustre: server umount lustre-MDT0000 complete [ 6635.940215] LustreError: 298190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572168 with bad export cookie 10476021141937735527 [ 6635.943273] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6635.943913] LustreError: 298190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6636.053631] Lustre: server umount lustre-MDT0001 complete [ 6638.857395] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6640.120313] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6641.657344] Lustre: server umount MGS complete [ 6645.280441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6645.411953] LustreError: 304278:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6645.431858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6646.735215] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6649.460529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6649.582899] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6650.826621] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6653.155252] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6655.032170] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6657.298132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6659.302986] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6661.774580] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6695.394807] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6695.396300] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6695.397110] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6695.397116] Lustre: Skipped 1 previous similar message [ 6695.398589] Lustre: Skipped 1 previous similar message [ 6700.110133] Lustre: server umount lustre-OST0000 complete [ 6705.634878] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6705.637095] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6705.640223] Lustre: Skipped 1 previous similar message [ 6705.642842] Lustre: Skipped 1 previous similar message [ 6707.484152] Lustre: server umount lustre-MDT0000 complete [ 6708.753187] LustreError: 304248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572241 with bad export cookie 10476021141937737340 [ 6708.756110] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6708.756550] LustreError: 304248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6708.885460] Lustre: server umount lustre-MDT0001 complete [ 6713.633850] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6715.468518] Key type lgssc unregistered [ 6715.694501] LNet: 308229:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6715.699182] LNetError: 308229:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6715.715442] LNet: Removed LNI 192.168.201.135@tcp [ 6716.274093] Key type .llcrypt unregistered [ 6716.275803] Key type ._llcrypt unregistered [ 6731.099770] Key type ._llcrypt registered [ 6731.102815] Key type .llcrypt registered [ 6731.198988] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 6740.832837] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6741.760426] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6741.789099] alg: No test for adler32 (adler32-zlib) [ 6742.725976] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 6742.882158] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6744.496185] Key type lgssc registered [ 6745.206804] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6759.703859] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6767.561585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6767.576761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6768.714579] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6768.728603] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6768.767410] Lustre: lustre-MDT0000: new disk, initializing [ 6768.808169] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6768.819144] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6770.768416] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6776.092268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6776.210103] Lustre: lustre-OST0000: new disk, initializing [ 6776.212339] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6776.214429] Lustre: Skipped 1 previous similar message [ 6776.217187] Lustre: 312255:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6776.244493] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6776.375565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 6776.384740] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 6776.403280] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6778.681626] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7019.822588] Lustre: server umount lustre-MDT0000 complete [ 7020.995174] LustreError: 311442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572553 with bad export cookie 6848589999216083369 [ 7020.998654] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7031.326407] Lustre: server umount lustre-OST0000 complete [ 7039.485230] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 7043.090264] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7055.439900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7055.540064] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7055.550389] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7055.582941] Lustre: lustre-MDT0000: new disk, initializing [ 7055.606118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7055.613753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7057.016614] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7061.278252] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7061.349167] Lustre: 316490:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7061.375171] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7061.377137] Lustre: Skipped 1 previous similar message [ 7061.418032] Lustre: lustre-MDT0001: new disk, initializing [ 7061.454000] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7061.466142] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7061.469419] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7062.844402] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7066.604984] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7068.455314] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7070.590962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7070.686646] Lustre: lustre-OST0000: new disk, initializing [ 7070.688381] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7070.690694] Lustre: 318459:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7070.717494] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7072.058219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7072.061634] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7072.094075] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7072.542724] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7076.122424] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7077.490540] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7077.556203] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7078.880691] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7078.952426] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7082.467289] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7082.467821] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7082.468522] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7082.477450] Lustre: Skipped 1 previous similar message [ 7085.710475] Lustre: server umount lustre-OST0000 complete [ 7087.072829] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7087.075508] LustreError: Skipped 1 previous similar message [ 7087.077120] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7087.081579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7087.084207] Lustre: Skipped 1 previous similar message [ 7092.705180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7092.707511] Lustre: Skipped 2 previous similar messages [ 7097.827437] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7097.830144] Lustre: Skipped 1 previous similar message [ 7101.408094] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7101.468360] Lustre: server umount lustre-MDT0000 complete [ 7102.608967] LustreError: 317287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572634 with bad export cookie 6848589999219024146 [ 7102.611324] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7102.611858] LustreError: 317287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7102.724360] Lustre: server umount lustre-MDT0001 complete [ 7103.821760] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7104.330853] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 18:10:35 (1786572635) [ 7106.494246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7106.687568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7107.993333] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7110.643507] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7112.031989] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7114.319970] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7116.126251] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7118.251957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7120.125056] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7122.442099] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7124.663893] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7124.723378] Lustre: lustre-OST0001: new disk, initializing [ 7124.725178] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7124.727971] Lustre: 323600:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7124.760021] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7124.762463] Lustre: Skipped 2 previous similar messages [ 7126.600658] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7128.837075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7128.840486] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7128.859917] LustreError: 324172:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7129.892341] LustreError: 324172:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7129.895563] LustreError: 324172:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 7130.915237] LustreError: 324176:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7130.919556] LustreError: 324176:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7132.963640] LustreError: 324172:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7132.967419] LustreError: 324172:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 7135.557870] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7135.627082] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7137.059544] LustreError: 324172:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7137.062747] LustreError: 324172:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 7137.364421] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8af2c5e43800.ost_server_uuid 50 [ 7137.882963] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8af2c5e43800.ost_server_uuid in FULL state after 0 sec [ 7139.612362] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8af2c5e43800.ost_server_uuid 50 [ 7140.078857] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8af2c5e43800.ost_server_uuid in FULL state after 0 sec [ 7141.162837] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7152.611641] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7152.613059] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7152.614507] Lustre: Skipped 2 previous similar messages [ 7157.693408] Lustre: server umount lustre-OST0001 complete [ 7157.730721] LustreError: 324179:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7157.734490] LustreError: 324179:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 7162.337196] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7162.339715] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7162.344129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7162.346744] Lustre: Skipped 2 previous similar messages [ 7162.851096] LustreError: 323131:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7162.856915] LustreError: 323131:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7165.331528] Lustre: server umount lustre-OST0000 complete [ 7166.944919] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7166.947740] LustreError: Skipped 1 previous similar message [ 7166.949545] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7166.952911] Lustre: Skipped 1 previous similar message [ 7172.761076] Lustre: server umount lustre-MDT0000 complete [ 7173.088979] LustreError: 320769:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7173.094902] LustreError: 320769:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7173.936075] LustreError: 320750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572706 with bad export cookie 6848589999219025014 [ 7173.938563] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7173.940144] LustreError: 320750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7174.061388] Lustre: server umount lustre-MDT0001 complete [ 7177.563189] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 7178.543430] Key type lgssc unregistered [ 7178.669281] LNet: 326290:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7178.671324] LNetError: 326290:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7178.677267] LNet: Removed LNI 192.168.201.135@tcp [ 7178.961564] Key type .llcrypt unregistered [ 7178.962698] Key type ._llcrypt unregistered [ 7187.739595] Key type ._llcrypt registered [ 7187.740351] Key type .llcrypt registered [ 7187.773711] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7188.139231] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7188.172734] alg: No test for adler32 (adler32-zlib) [ 7189.028723] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 7189.113045] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 7190.696135] Key type lgssc registered [ 7191.045458] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7193.491433] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7193.498498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7194.635419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7195.969884] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7198.348682] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7200.538893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7200.654130] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7201.959185] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7204.911239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7205.021662] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7206.859414] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7209.064620] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7211.124900] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7211.196215] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7212.981205] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7215.139621] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7219.689825] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 7219.691387] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7222.067672] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 7222.139483] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7298.529225] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7298.531108] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7298.534814] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7299.557212] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7299.557409] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7299.563022] Lustre: Skipped 1 previous similar message [ 7302.715895] Lustre: server umount lustre-OST0001 complete [ 7304.674419] LustreError: 329363:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7304.674492] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7304.676889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7304.680766] LustreError: 329363:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7304.687839] Lustre: Skipped 1 previous similar message [ 7309.795156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7309.795168] LustreError: 329363:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7309.798461] Lustre: Skipped 2 previous similar messages [ 7309.805317] LustreError: 329363:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7310.413434] Lustre: server umount lustre-OST0000 complete [ 7313.376943] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7313.379976] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7313.826985] Lustre: server umount lustre-MDT0000 complete [ 7314.913161] LustreError: 327687:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7314.920219] LustreError: 327687:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7315.125140] LustreError: 327667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572847 with bad export cookie 16085724384850972388 [ 7315.129506] LustreError: 327667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7315.129994] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7315.257298] Lustre: server umount lustre-MDT0001 complete [ 7318.918548] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 7319.864700] Key type lgssc unregistered [ 7319.993417] LNet: 332622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7319.995309] LNetError: 332622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7320.002268] LNet: Removed LNI 192.168.201.135@tcp [ 7320.296127] Key type .llcrypt unregistered [ 7320.296935] Key type ._llcrypt unregistered [ 7337.743634] Key type ._llcrypt registered [ 7337.744612] Key type .llcrypt registered [ 7337.797283] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7338.214702] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7338.281387] alg: No test for adler32 (adler32-zlib) [ 7339.164520] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 7339.267382] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 7340.848209] Key type lgssc registered [ 7341.244447] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7344.420082] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7344.424890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7345.561977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7346.891465] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7349.698778] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7349.829019] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7351.213266] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7352.236707] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7354.547197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7354.679447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7355.684799] LustreError: 336377:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7355.688833] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 7355.692612] LustreError: 336377:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7356.663466] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7359.523180] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7359.602136] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7360.620736] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7361.550393] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7365.936072] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7371.624545] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7375.328910] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7375.332473] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7375.338466] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7376.355902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7376.356015] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7376.357740] Lustre: Skipped 1 previous similar message [ 7376.362309] Lustre: Skipped 2 previous similar messages [ 7377.249650] Lustre: server umount lustre-MDT0000 complete [ 7378.429490] LustreError: 334869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572910 with bad export cookie 8128200399150794712 [ 7378.431876] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7378.433755] LustreError: 334869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7380.590246] Lustre: server umount lustre-MDT0001 complete [ 7392.141246] Lustre: server umount lustre-OST0000 complete [ 7403.619156] Lustre: server umount lustre-OST0001 complete [ 7408.654892] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7412.004921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7412.135460] LustreError: 340145:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7412.152708] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7413.484765] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7416.314190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7417.809984] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7418.728046] Lustre: 341285:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7420.789364] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7420.903270] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7420.905234] Lustre: Skipped 1 previous similar message [ 7421.923254] LustreError: 341638:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7421.926367] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 7422.709923] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7425.415215] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7426.539620] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7427.313601] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7430.293521] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7431.385392] Lustre: 343156:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7436.770560] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7436.774119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7436.775246] Lustre: Skipped 3 previous similar messages [ 7436.776850] Lustre: Skipped 2 previous similar messages [ 7438.369229] Lustre: server umount lustre-MDT0000 complete [ 7439.562104] LustreError: 340126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572971 with bad export cookie 8128200399150796371 [ 7439.564653] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7439.565739] LustreError: 340126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7439.677190] Lustre: server umount lustre-MDT0001 complete [ 7451.141718] Lustre: server umount lustre-OST0000 complete [ 7462.622255] Lustre: server umount lustre-OST0001 complete [ 7463.773953] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 7464.274301] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 7464.830448] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 18:16:36 (1786572996) [ 7471.827475] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7474.592367] loop0: detected capacity change from 0 to 1099511627776 [ 7474.738804] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 7474.741296] *** VALIDATE xfs *** [ 7474.744820] XFS (loop0): Mounting V5 Filesystem [ 7474.752121] XFS (loop0): Ending clean mount [ 7475.152051] loop1: detected capacity change from 0 to 18691697672192 [ 7496.606390] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 7500.191221] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7512.107560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7512.192613] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7512.203267] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7512.240530] Lustre: lustre-MDT0000: new disk, initializing [ 7512.263255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7512.265207] Lustre: Skipped 1 previous similar message [ 7512.270021] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7513.725725] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7517.977800] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7518.026571] Lustre: 348267:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7518.051469] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7518.054603] Lustre: Skipped 1 previous similar message [ 7518.095810] Lustre: lustre-MDT0001: new disk, initializing [ 7518.143323] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7518.148528] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7519.597706] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7523.390455] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7525.358430] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7527.620129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7527.721184] Lustre: lustre-OST0000: new disk, initializing [ 7527.723726] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7527.727176] Lustre: 350236:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7529.676502] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7529.787113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7529.791309] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7529.820336] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7533.471686] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7535.042902] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7535.124893] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7536.607576] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7536.686075] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7540.193807] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7540.195172] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7540.196360] LustreError: Skipped 1 previous similar message [ 7540.200410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7540.203358] Lustre: Skipped 1 previous similar message [ 7543.512298] Lustre: server umount lustre-OST0000 complete [ 7545.313759] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7545.317618] Lustre: Skipped 1 previous similar message [ 7545.319467] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7545.321144] Lustre: Skipped 1 previous similar message [ 7555.553136] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7555.554975] Lustre: Skipped 3 previous similar messages [ 7559.136111] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7559.215808] Lustre: server umount lustre-MDT0000 complete [ 7559.649248] LustreError: 349078:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7559.656082] LustreError: 349078:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7560.520339] LustreError: 348258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786573092 with bad export cookie 8128200399150797904 [ 7560.523564] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7560.523874] LustreError: 348258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7560.640164] Lustre: server umount lustre-MDT0001 complete [ 7563.438259] XFS (loop0): Unmounting Filesystem [ 7565.108402] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 18:18:16 (1786573096) [ 7567.345826] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7567.495171] LustreError: 352790:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7567.520787] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7567.522721] Lustre: Skipped 2 previous similar messages [ 7568.825841] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7571.552471] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7573.081301] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7575.548514] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7577.361916] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7579.574508] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7581.577426] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7584.006089] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7602.146588] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7602.147768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7602.150094] LustreError: Skipped 1 previous similar message [ 7602.153655] Lustre: Skipped 1 previous similar message [ 7602.154586] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7602.159090] Lustre: Skipped 1 previous similar message [ 7603.673423] Lustre: server umount lustre-OST0000 complete [ 7611.032196] Lustre: server umount lustre-MDT0000 complete [ 7612.271186] LustreError: 352771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786573144 with bad export cookie 8128200399150798786 [ 7612.274923] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7612.275027] LustreError: 352771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7612.393406] Lustre: server umount lustre-MDT0001 complete [ 7616.234458] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 7617.284576] Key type lgssc unregistered [ 7617.415431] LNet: 356439:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7617.418151] LNetError: 356439:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7617.424339] LNet: Removed LNI 192.168.201.135@tcp [ 7617.735127] Key type .llcrypt unregistered [ 7617.736144] Key type ._llcrypt unregistered [ 7627.940329] Key type ._llcrypt registered [ 7627.941352] Key type .llcrypt registered [ 7627.981588] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7628.289948] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7628.299636] alg: No test for adler32 (adler32-zlib) [ 7629.159556] Lustre: Lustre: Build Version: 2.17.57_1_gc4b410f [ 7629.250268] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 7630.832181] Key type lgssc registered [ 7631.197840] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7633.803905] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7633.810692] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7634.952335] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7636.260727] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7639.142389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7639.276590] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7640.662898] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7643.137411] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7644.926349] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7647.069844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7647.181943] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7649.125932] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7651.511054] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7665.121219] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7665.124036] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7665.129868] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7666.658084] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7666.659715] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7670.738991] Lustre: server umount lustre-MDT0001 complete [ 7671.777393] LustreError: 360633:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7671.781819] LustreError: 360633:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7672.327078] LustreError: 358716:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7674.238919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7674.269895] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7674.271396] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7674.282957] Lustre: 357973:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7674.397212] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7675.800439] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7679.463442] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7679.469249] LustreError: 361666:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7679.472529] LustreError: 361666:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7679.475068] LustreError: 357975:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7679.492479] LustreError: 361669:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -17 [ 7679.496676] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 7679.496676] [ 7979.531860] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.35@tcp (stopping) [ 7979.533878] Lustre: Skipped 2 previous similar messages [ 7981.541237] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7981.541346] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7981.549452] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7981.552368] Lustre: Skipped 1 previous similar message [ 7984.342694] Lustre: server umount lustre-MDT0001 complete [ 7984.645849] LustreError: 357978:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7986.657275] LustreError: 357983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7986.661319] LustreError: 357983:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7987.636148] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7987.668552] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7987.670218] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7987.683078] Lustre: 357971:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7987.786765] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7989.096809] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7992.806148] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7996.902097] LustreError: 374489:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7996.904445] LustreError: 374489:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7996.906809] LustreError: 357975:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7996.908489] LustreError: 357975:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7996.924135] LustreError: 374492:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 7996.926076] LustreError: 374492:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 7996.927653] LustreError: 374492:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -17 [ 7996.929947] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.135@tcp [ 7996.929947] [ 8292.834037] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8292.836804] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8292.846878] Lustre: Skipped 2 previous similar messages [ 8298.486107] Lustre: server umount lustre-MDT0001 complete [ 8302.103605] LustreError: 358716:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8303.768468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8303.831691] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8303.834950] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8303.852482] Lustre: 357971:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8303.977234] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 8305.797598] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8309.225472] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8311.783778] LustreError: 386563:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8311.787532] LustreError: 386563:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8311.791407] LustreError: 357975:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8311.795190] LustreError: 357975:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8311.815304] LustreError: 386566:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 8311.822295] LustreError: 386566:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -114 [ 8311.826830] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 8311.826830] [ 8621.538209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8621.540323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8621.542938] Lustre: Skipped 2 previous similar messages [ 8621.544586] Lustre: Skipped 5 previous similar messages [ 8621.545849] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8626.588264] Lustre: server umount lustre-MDT0000 complete [ 8626.658123] LustreError: 357984:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8626.662213] LustreError: 357984:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 8627.855379] LustreError: 360632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786574160 with bad export cookie 14577318762650238206 [ 8627.858352] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8627.859838] LustreError: 360632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8631.777449] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8634.053230] Lustre: server umount lustre-MDT0001 complete [ 8635.546314] Lustre: server umount lustre-OST0000 complete [ 8638.953912] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 8642.419393] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8655.490683] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:36:26 (1786574186) [ 8657.796890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8657.884202] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8657.895847] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8657.931342] Lustre: lustre-MDT0000: new disk, initializing [ 8657.955228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8657.963784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8659.376509] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8663.653952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8663.692400] Lustre: 400725:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8663.713721] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8663.715482] Lustre: Skipped 1 previous similar message [ 8663.749899] Lustre: lustre-MDT0001: new disk, initializing [ 8663.783168] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8663.795749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8663.798594] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8665.097703] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8668.730233] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8670.512432] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8672.840878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8672.937570] Lustre: lustre-OST0000: new disk, initializing [ 8672.940206] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8672.944215] Lustre: 402696:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8672.976583] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8674.967691] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8678.763232] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8680.441621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8680.445029] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8680.484057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8689.632997] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8689.635786] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8689.641969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8689.643576] Lustre: Skipped 2 previous similar messages [ 8692.706488] Lustre: server umount lustre-MDT0000 complete [ 8694.028233] LustreError: 400718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786574226 with bad export cookie 14577318762650240145 [ 8694.030636] LustreError: 401536:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 8694.032490] LustreError: 400718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8694.032863] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8694.158988] Lustre: server umount lustre-MDT0001 complete [ 8704.736179] Lustre: server umount lustre-OST0000 complete [ 8714.714464] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 8717.877903] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8728.172527] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 8384 sec ========== 18:37:39 (1786574259) [ 8728.725604] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:37:40 (1786574260) === [ 8729.810198] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:37:41 (1786574261) === [ 8743.166050] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8744.193283] Key type lgssc unregistered [ 8744.332323] LNet: 409053:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8744.335086] LNetError: 409053:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8744.343293] LNet: Removed LNI 192.168.201.135@tcp [ 8744.661576] Key type .llcrypt unregistered [ 8744.662834] Key type ._llcrypt unregistered