[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 495525741 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524596K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002010] Switched APIC routing to physical x2apic. [ 0.003013] kvm-guest: setup PV IPIs [ 0.006575] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008016] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009010] pid_max: default: 32768 minimum: 301 [ 0.011025] LSM: Security Framework initializing [ 0.012064] Yama: becoming mindful. [ 0.013039] SELinux: Initializing. [ 0.014082] *** VALIDATE selinux *** [ 0.023449] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027785] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028150] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029103] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031082] *** VALIDATE tmpfs *** [ 0.032408] *** VALIDATE proc *** [ 0.034219] *** VALIDATE cgroup *** [ 0.035010] *** VALIDATE cgroup2 *** [ 0.036270] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037152] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039034] Spectre V2 : User space: Vulnerable [ 0.040009] Speculative Store Bypass: Vulnerable [ 0.043302] debug: unmapping init [mem 0xffffffffa5659000-0xffffffffa5660fff] [ 0.046108] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047711] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048023] ... version: 2 [ 0.049011] ... bit width: 48 [ 0.050011] ... generic registers: 4 [ 0.051010] ... value mask: 0000ffffffffffff [ 0.052009] ... max period: 00007fffffffffff [ 0.053010] ... fixed-purpose events: 3 [ 0.054011] ... event mask: 000000070000000f [ 0.055258] rcu: Hierarchical SRCU implementation. [ 0.057322] smp: Bringing up secondary CPUs ... [ 0.058572] x86: Booting SMP configuration: [ 0.059029] .... node #0, CPUs: #1 #2 #3 [ 0.062407] smp: Brought up 1 node, 4 CPUs [ 0.064012] smpboot: Max logical packages: 1 [ 0.065015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.157031] node 0 deferred pages initialised in 89ms [ 0.161216] devtmpfs: initialized [ 0.162376] x86/mm: Memory block size: 128MB [ 0.164603] gcov: version magic: 0x41383552 [ 0.167274] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.168095] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.169253] pinctrl core: initialized pinctrl subsystem [ 0.170190] [ 0.170946] ************************************************************* [ 0.171017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.172013] ** ** [ 0.173013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.174014] ** ** [ 0.175015] ** This means that this kernel is built to expose internal ** [ 0.176016] ** IOMMU data structures, which may compromise security on ** [ 0.177016] ** your system. ** [ 0.178012] ** ** [ 0.179011] ** If you see this message and you are not debugging the ** [ 0.180014] ** kernel, report this immediately to your vendor! ** [ 0.181015] ** ** [ 0.182013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.183011] ************************************************************* [ 0.184877] NET: Registered protocol family 16 [ 0.188485] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.191082] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.194167] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.199039] cpuidle: using governor menu [ 0.201173] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.204889] PCI: Using configuration type 1 for base access [ 0.207146] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.219219] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.222031] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.225172] cryptd: max_cpu_qlen set to 1000 [ 0.228332] ACPI: Added _OSI(Module Device) [ 0.231026] ACPI: Added _OSI(Processor Device) [ 0.233023] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.235024] ACPI: Added _OSI(Processor Aggregator Device) [ 0.240802] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.253557] ACPI: Interpreter enabled [ 0.256120] ACPI: PM: (supports S0 S3 S4 S5) [ 0.257020] ACPI: Using IOAPIC for interrupt routing [ 0.260169] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.264515] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.276118] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.279070] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.282027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.288104] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.293568] acpiphp: Slot [2] registered [ 0.296170] acpiphp: Slot [5] registered [ 0.297113] acpiphp: Slot [6] registered [ 0.299141] acpiphp: Slot [7] registered [ 0.303241] acpiphp: Slot [8] registered [ 0.305233] acpiphp: Slot [9] registered [ 0.307206] acpiphp: Slot [10] registered [ 0.310166] acpiphp: Slot [3] registered [ 0.311154] acpiphp: Slot [4] registered [ 0.313148] acpiphp: Slot [11] registered [ 0.315149] acpiphp: Slot [12] registered [ 0.317157] acpiphp: Slot [13] registered [ 0.318155] acpiphp: Slot [14] registered [ 0.320141] acpiphp: Slot [15] registered [ 0.322214] acpiphp: Slot [16] registered [ 0.323133] acpiphp: Slot [17] registered [ 0.325159] acpiphp: Slot [18] registered [ 0.327152] acpiphp: Slot [19] registered [ 0.328157] acpiphp: Slot [20] registered [ 0.330168] acpiphp: Slot [21] registered [ 0.331227] acpiphp: Slot [22] registered [ 0.333148] acpiphp: Slot [23] registered [ 0.335139] acpiphp: Slot [24] registered [ 0.336138] acpiphp: Slot [25] registered [ 0.338206] acpiphp: Slot [26] registered [ 0.340190] acpiphp: Slot [27] registered [ 0.341146] acpiphp: Slot [28] registered [ 0.343145] acpiphp: Slot [29] registered [ 0.344133] acpiphp: Slot [30] registered [ 0.346140] acpiphp: Slot [31] registered [ 0.348098] PCI host bridge to bus 0000:00 [ 0.349027] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.352042] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.355040] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.358033] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.360030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.364047] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.366221] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.369148] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.373525] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.384921] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.388962] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.392028] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.395029] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.399034] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.402870] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.406073] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.409057] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.411877] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.418022] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.432046] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.440018] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.445447] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.457074] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.464016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.482017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.492096] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.499021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.510023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.532022] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.546240] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.553016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.561014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.580022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.592171] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.597022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.602016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.619032] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.628230] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.641023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.665021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.686016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.701480] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.709014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.716017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.739023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.756005] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.758401] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.761381] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.764569] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.767318] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.773117] iommu: Default domain type: Passthrough [ 0.775508] SCSI subsystem initialized [ 0.776200] ACPI: bus type USB registered [ 0.778166] usbcore: registered new interface driver usbfs [ 0.780109] usbcore: registered new interface driver hub [ 0.782114] usbcore: registered new device driver usb [ 0.783000] pps_core: LinuxPPS API ver. 1 registered [ 0.783017] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.787052] PTP clock support registered [ 0.791102] EDAC MC: Ver: 3.0.0 [ 0.793138] PCI: Using ACPI for IRQ routing [ 0.796358] NetLabel: Initializing [ 0.798011] NetLabel: domain hash size = 128 [ 0.800010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.802098] NetLabel: unlabeled traffic allowed by default [ 0.805335] vgaarb: loaded [ 0.806259] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.810023] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.817372] clocksource: Switched to clocksource kvm-clock [ 0.932582] VFS: Disk quotas dquot_6.6.0 [ 0.934149] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.938250] *** VALIDATE ramfs *** [ 0.939675] *** VALIDATE hugetlbfs *** [ 0.941371] pnp: PnP ACPI init [ 0.943675] pnp: PnP ACPI: found 6 devices [ 0.960904] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.964679] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.967800] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.969459] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.971455] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.973337] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.975568] NET: Registered protocol family 2 [ 0.977565] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.981840] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.984612] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.992339] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.996730] TCP: Hash tables configured (established 65536 bind 65536) [ 1.000274] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.004042] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.007276] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.012147] NET: Registered protocol family 1 [ 1.014778] RPC: Registered named UNIX socket transport module. [ 1.017693] RPC: Registered udp transport module. [ 1.019856] RPC: Registered tcp transport module. [ 1.021586] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.024297] NET: Registered protocol family 44 [ 1.026201] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.028724] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.031141] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.033955] PCI: CLS 0 bytes, default 64 [ 1.035484] Unpacking initramfs... [ 2.489880] debug: unmapping init [mem 0xffff8b8a7cc54000-0xffff8b8a7ffbffff] [ 2.494509] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.497054] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.500351] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.658650] Initialise system trusted keyrings [ 3.661551] Key type blacklist registered [ 3.665529] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.708258] zbud: loaded [ 3.720236] *** VALIDATE nfs *** [ 3.728213] *** VALIDATE nfs4 *** [ 3.730261] pstore: using deflate compression [ 3.758803] Platform Keyring initialized [ 4.040244] NET: Registered protocol family 38 [ 4.043525] Key type asymmetric registered [ 4.047810] Asymmetric key parser 'x509' registered [ 4.051753] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.057473] io scheduler mq-deadline registered [ 4.063708] io scheduler kyber registered [ 4.068949] io scheduler bfq registered [ 4.073190] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.088728] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.100549] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.106075] ACPI: Power Button [PWRF] [ 4.124017] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.151261] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.187273] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.202676] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.250050] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.298864] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.339133] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.359780] Non-volatile memory driver v1.3 [ 4.366714] Linux agpgart interface v0.103 [ 4.505280] virtio_blk virtio1: [vda] 146016 512-byte logical blocks (74.8 MB/71.3 MiB) [ 4.519259] vda: detected capacity change from 0 to 74760192 [ 4.568686] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.579741] vdb: detected capacity change from 0 to 1073741824 [ 4.645855] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.652619] vdc: detected capacity change from 0 to 2621440000 [ 4.715737] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.722461] vdd: detected capacity change from 0 to 2621440000 [ 4.758358] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.765130] vde: detected capacity change from 0 to 4294967296 [ 4.807260] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.814496] vdf: detected capacity change from 0 to 4294967296 [ 4.835832] libphy: Fixed MDIO Bus: probed [ 4.850518] usbcore: registered new interface driver usbserial_generic [ 4.854416] usbserial: USB Serial support registered for generic [ 4.857261] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.862866] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.865263] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.868215] mousedev: PS/2 mouse device common for all mice [ 4.872831] rtc_cmos 00:05: RTC can wake from S4 [ 4.876507] rtc_cmos 00:05: registered as rtc0 [ 4.878441] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.880734] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.885953] intel_pstate: CPU model not supported [ 4.890197] hid: raw HID events driver (C) Jiri Kosina [ 4.897983] usbcore: registered new interface driver usbhid [ 4.900797] usbhid: USB HID core driver [ 4.904458] drop_monitor: Initializing network drop monitor service [ 4.909712] Initializing XFRM netlink socket [ 4.912939] NET: Registered protocol family 10 [ 4.914524] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.921199] Segment Routing with IPv6 [ 4.928762] NET: Registered protocol family 17 [ 4.931547] mpls_gso: MPLS GSO support [ 4.936143] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.942553] RAS: Correctable Errors collector initialized. [ 4.947384] AVX version of gcm_enc/dec engaged. [ 4.949667] AES CTR mode by8 optimization enabled [ 5.125251] sched_clock: Marking stable (5125125224, 0)->(6049457599, -924332375) [ 5.130388] registered taskstats version 1 [ 5.133200] Loading compiled-in X.509 certificates [ 5.135526] zswap: loaded using pool lzo/zbud [ 5.187881] Key type big_key registered [ 5.234888] Key type encrypted registered [ 5.244611] ima: No TPM chip found, activating TPM-bypass! [ 5.264198] ima: Allocated hash algorithm: sha1 [ 5.269150] ima: No architecture policies found [ 5.275321] evm: Initialising EVM extended attributes: [ 5.281667] evm: security.selinux [ 5.283649] evm: security.ima [ 5.287226] evm: security.capability [ 5.296241] evm: HMAC attrs: 0x1 [ 5.308685] rtc_cmos 00:05: setting system clock to 2026-08-31 04:58:14 UTC (1788152294) [ 5.328066] debug: unmapping init [mem 0xffffffffa6603000-0xffffffffa67fffff] [ 5.339754] debug: unmapping init [mem 0xffffffffa5382000-0xffffffffa5658fff] [ 5.359105] Write protecting the kernel read-only data: 28672k [ 5.373792] debug: unmapping init [mem 0xffffffffa3a03000-0xffffffffa3bfffff] [ 5.385115] debug: unmapping init [mem 0xffffffffa4314000-0xffffffffa43fffff] [ 5.512347] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.534979] systemd[1]: Detected virtualization kvm. [ 5.538428] systemd[1]: Detected architecture x86-64. [ 5.543713] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.597465] systemd[1]: No hostname configured. [ 5.602400] systemd[1]: Set hostname to . [ 5.604786] random: systemd: uninitialized urandom read (16 bytes read) [ 5.610181] systemd[1]: Initializing machine ID from random generator. [ 5.731142] random: ln: uninitialized urandom read (6 bytes read) [ 6.002380] random: systemd: uninitialized urandom read (16 bytes read) [ 6.007351] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 6.019352] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 6.042243] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Timers. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Paths. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.823331] device-mapper: uevent: version 1.0.3 [ 7.832780] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 10.090857] virtio_net virtio0 ens2: renamed from eth0 [ 10.173400] random: fast init done [ 10.269692] scsi host0: ata_piix [ 10.457473] scsi host1: ata_piix [ 10.461034] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 10.463972] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 14.818911] random: crng init done [ 14.821548] random: 7 urandom warning(s) missed due to ratelimiting [ 18.068115] hrtimer: interrupt took 7921208 ns [ 18.093375] dracut-initqueue[592]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 19.834817] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 23.326309] printk: systemd: 26 output lines suppressed due to ratelimiting [ 24.489685] SELinux: Disabled at runtime. [ 24.655594] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 24.676870] systemd[1]: Detected virtualization kvm. [ 24.682703] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 26.609359] systemd[1]: initrd-switch-root.service: Succeeded. [ 26.633684] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 26.656315] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 26.663219] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 26.669328] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 26.707788] systemd[1]: Starting Journal Service... Starting Journal Service... [ 26.760097] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Remount Root and Kernel File Systems... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. [ OK ] Stopped target Switch Root. Mounting Huge Pages File System... Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... Mounting POSIX Message Queue File System... [ 27.207370] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 28.398420] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 29.602279] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 29.852503] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 30.384667] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 30.484815] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 36.014534] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 37.119232] NFS: Registering the id_resolver key type [ 37.125783] Key type id_resolver registered [ 37.127889] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ ***] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg625-server login: [ 104.357475] libcfs: loading out-of-tree module taints kernel. [ 104.408485] Key type ._llcrypt registered [ 104.411613] Key type .llcrypt registered [ 104.491433] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 125.444075] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 127.006848] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 127.019587] alg: No test for adler32 (adler32-zlib) [ 128.539876] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 129.374049] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 131.079447] Key type lgssc registered [ 132.893932] Lustre: Echo OBD driver; http://www.lustre.org/ [ 150.638584] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 196.218969] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 209.063896] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 209.157464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 210.410269] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 210.443424] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 210.549612] Lustre: lustre-MDT0000: new disk, initializing [ 210.630906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 210.655957] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 214.922551] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 229.938671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 230.052917] Lustre: 6529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 230.100208] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 230.107761] Lustre: Skipped 1 previous similar message [ 230.263275] Lustre: lustre-MDT0001: new disk, initializing [ 230.328457] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 230.373382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 230.386116] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 234.996318] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 240.518542] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 250.405441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 250.608764] Lustre: lustre-OST0000: new disk, initializing [ 250.613442] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 250.618513] Lustre: 8470:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 250.692724] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 256.021794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 256.044197] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 256.085966] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 256.748897] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 270.371785] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 270.537802] Lustre: lustre-OST0001: new disk, initializing [ 270.540874] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 270.543900] Lustre: 9541:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 270.603575] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 277.091427] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 278.059146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 278.087115] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 278.116600] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 289.101997] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 300.101300] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 306.927944] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing check_logdir /tmp/testlogs/ [ 311.734381] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing yml_node [ 316.551175] Lustre: DEBUG MARKER: Client: 2.17.55.27 [ 319.339651] Lustre: DEBUG MARKER: MDS: 2.17.55.27 [ 322.474635] Lustre: DEBUG MARKER: OSS: 2.17.55.27 [ 324.484738] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 31 01:03:32 EDT 2026 [ 341.773894] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 343.518588] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 363.493209] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 363.509986] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 363.552724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 365.027820] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 365.029542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 365.051095] Lustre: Skipped 2 previous similar messages [ 365.090177] Lustre: Skipped 3 previous similar messages [ 366.716282] Lustre: server umount lustre-MDT0000 complete [ 371.058942] LustreError: 6521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788152660 with bad export cookie 5213298497284591284 [ 371.061176] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 371.078147] LustreError: 6521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 375.266742] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 375.289109] LustreError: 6538:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 375.294450] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 375.312717] LustreError: 6538:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 375.332878] Lustre: Skipped 1 previous similar message [ 376.289166] LustreError: 6536:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 376.313122] LustreError: 6536:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 377.670333] Lustre: server umount lustre-MDT0001 complete [ 381.560588] Lustre: server umount lustre-OST0000 complete [ 385.364990] Lustre: server umount lustre-OST0001 complete [ 392.655108] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 404.812668] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 445.471697] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 445.800801] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 445.856913] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 445.932838] Lustre: lustre-MDT0000: new disk, initializing [ 446.041748] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 446.066977] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 451.314318] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 461.645048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 461.778470] Lustre: 15099:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 461.856696] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 461.865504] Lustre: Skipped 1 previous similar message [ 461.958080] Lustre: lustre-MDT0001: new disk, initializing [ 462.091508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 462.161619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 462.186260] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 466.578392] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 476.074527] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 481.945347] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 488.603410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 489.007848] Lustre: lustre-OST0000: new disk, initializing [ 489.010981] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 489.016539] Lustre: 17069:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 489.113588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 490.264777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 490.278780] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 490.485559] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 495.912313] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 505.700527] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 510.439506] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 510.644415] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 514.819149] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 515.012960] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 521.187656] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 521.195159] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 521.199206] LustreError: Skipped 1 previous similar message [ 521.201970] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 521.201979] Lustre: Skipped 2 previous similar messages [ 521.232734] Lustre: Skipped 2 previous similar messages [ 523.136560] Lustre: server umount lustre-OST0000 complete [ 528.866772] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 528.877728] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 528.893725] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 528.904421] Lustre: Skipped 1 previous similar message [ 541.153036] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 541.396855] Lustre: server umount lustre-MDT0000 complete [ 541.665030] LustreError: 15105:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.683111] LustreError: 15105:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 544.971553] LustreError: 15895:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788152834 with bad export cookie 5213298497284592817 [ 544.979327] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 544.982294] LustreError: 15895:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 545.469768] Lustre: server umount lustre-MDT0001 complete [ 550.756463] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:07:18 (1788152838) [ 556.682872] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 557.197495] LustreError: 19333:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.260827] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 561.955975] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 562.657918] LustreError: 19334:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 569.886562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 574.154727] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 580.782974] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 586.553403] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 592.957383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 598.758605] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 606.349903] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 609.785769] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 616.419876] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 616.429370] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 616.437300] Lustre: Skipped 1 previous similar message [ 616.443494] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 616.449973] Lustre: Skipped 4 previous similar messages [ 621.427634] Lustre: server umount lustre-MDT0000 complete [ 623.074076] LustreError: 19334:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 623.108604] LustreError: 19334:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 625.612799] LustreError: 19315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788152914 with bad export cookie 5213298497284593692 [ 625.618801] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 625.631971] LustreError: 19315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 626.079611] Lustre: server umount lustre-MDT0001 complete [ 640.814613] Lustre: server umount lustre-OST0000 complete [ 660.265679] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 669.814594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 670.217647] LustreError: 23713:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 670.274612] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 670.279815] Lustre: Skipped 2 previous similar messages [ 674.014325] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 682.654468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 688.567231] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 692.528767] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 700.884753] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 708.403527] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 716.581476] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 716.708596] Lustre: lustre-OST0001: new disk, initializing [ 716.717556] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 716.727093] Lustre: 26035:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 718.748910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 718.769966] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 718.919209] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 722.932835] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 732.926753] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 736.853338] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 749.542393] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 749.546778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 749.558521] Lustre: Skipped 5 previous similar messages [ 749.558898] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 749.565487] Lustre: Skipped 6 previous similar messages [ 754.670620] LustreError: 23709:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 754.683909] LustreError: 23709:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 754.792119] Lustre: server umount lustre-MDT0000 complete [ 758.305231] LustreError: 23693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788153047 with bad export cookie 5213298497284594546 [ 758.315576] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 758.339778] LustreError: 23693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 758.637477] Lustre: server umount lustre-MDT0001 complete [ 772.821768] Lustre: server umount lustre-OST0000 complete [ 775.136655] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788153048/real 1788153048] req@ffff8b89c7cb3480 x1875013710101888/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788153064 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 776.822888] Lustre: server umount lustre-OST0001 complete [ 792.851804] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 801.769778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 802.071870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 802.078591] Lustre: Skipped 3 previous similar messages [ 805.699313] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 813.272163] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 817.572732] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 820.532709] Lustre: 30205:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 826.786352] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 828.220343] LustreError: 30558:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 828.254731] LustreError: 30558:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 833.850157] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 841.881609] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 847.940261] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 855.329711] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 858.261924] Lustre: 32082:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 864.736302] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 864.751731] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 864.765274] Lustre: Skipped 1 previous similar message [ 864.775617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 870.251059] Lustre: server umount lustre-MDT0000 complete [ 873.918378] LustreError: 29036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788153163 with bad export cookie 5213298497284595841 [ 873.934703] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 873.938192] LustreError: 29036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 874.310471] Lustre: server umount lustre-MDT0001 complete [ 888.459466] Lustre: server umount lustre-OST0000 complete [ 902.429907] Lustre: server umount lustre-OST0001 complete [ 910.341767] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:13:18 (1788153198) [ 939.114405] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 949.791110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 955.169161] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 959.984269] LustreError: 34886:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 960.022086] LustreError: 34886:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 963.501796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 967.762264] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 970.448463] Lustre: 36033:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 976.529636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 982.779469] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 991.261335] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 997.241728] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1004.371547] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1007.706194] Lustre: 37914:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1015.265074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1015.278469] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1015.301409] Lustre: Skipped 3 previous similar messages [ 1015.316895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1015.336425] Lustre: Skipped 4 previous similar messages [ 1018.590103] Lustre: server umount lustre-MDT0000 complete [ 1022.196392] LustreError: 34865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788153311 with bad export cookie 5213298497284596947 [ 1022.204824] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1022.206689] LustreError: 34865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1022.617799] Lustre: server umount lustre-MDT0001 complete [ 1036.910249] Lustre: server umount lustre-OST0000 complete [ 1050.780458] Lustre: server umount lustre-OST0001 complete [ 1058.305821] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:15:46 (1788153346) [ 1072.860478] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 1081.796799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1082.178503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1082.181128] Lustre: Skipped 7 previous similar messages [ 1085.591374] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1092.886417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1096.999932] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1099.480189] Lustre: 41206:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1104.839809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1110.959692] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1118.379100] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1123.774377] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1129.644399] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1132.977735] Lustre: 43085:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1133.687481] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1133.692862] Lustre: Skipped 1 previous similar message [ 1139.168882] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1142.455973] Lustre: server umount lustre-MDT0000 complete [ 1146.796046] LustreError: 40037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788153435 with bad export cookie 5213298497284598053 [ 1146.797044] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1146.812055] LustreError: 40037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1150.435721] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1150.455469] Lustre: Skipped 7 previous similar messages [ 1150.466933] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 1150.477064] Lustre: Skipped 12 previous similar messages [ 1161.183394] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1161.446644] Lustre: server umount lustre-MDT0001 complete [ 1169.994348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1177.079644] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1186.005334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1187.374523] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1191.204502] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1199.766292] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1201.591555] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1207.789613] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1209.630780] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1217.009499] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1220.813960] Lustre: server umount lustre-MDT0000 complete [ 1223.155243] LustreError: 45221:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1223.195931] LustreError: 45221:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 33 previous similar messages [ 1225.092154] LustreError: 40039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788153514 with bad export cookie 5213298497284599145 [ 1225.102034] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1225.109161] LustreError: 40039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1225.365640] Lustre: server umount lustre-MDT0001 complete [ 1241.794733] Lustre: server umount lustre-OST0000 complete [ 1244.512041] Lustre: 3651:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788153517/real 1788153517] req@ffff8b8aff49d180 x1875013710285184/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788153533 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1246.582426] Lustre: server umount lustre-OST0001 complete [ 1256.597588] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:19:04 (1788153544) [ 1276.295247] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 1287.118920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1292.061362] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1301.319867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1306.167941] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1308.958701] Lustre: 48845:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1315.414852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1321.741786] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1330.756068] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1336.535962] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1343.292878] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1347.249415] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1358.308945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1360.402994] Lustre: server umount lustre-MDT0000 complete [ 1363.782681] LustreError: 47672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788153652 with bad export cookie 5213298497284599971 [ 1363.793528] LustreError: 47672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1363.797109] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1364.117088] Lustre: server umount lustre-MDT0001 complete [ 1377.417065] Lustre: server umount lustre-OST0000 complete [ 1390.264885] Lustre: server umount lustre-OST0001 complete [ 1399.596703] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:21:27 (1788153687) [ 1401.565482] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1403.177213] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:21:31 (1788153691) [ 1414.911863] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 1424.446953] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 1445.521070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1445.846751] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1445.953283] Lustre: lustre-MDT0000: new disk, initializing [ 1446.051610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1450.224855] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1459.206299] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1465.393544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1465.639188] Lustre: lustre-OST0000: new disk, initializing [ 1465.643616] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1465.649422] Lustre: Skipped 1 previous similar message [ 1465.655815] Lustre: 54921:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1467.231978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1467.241707] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1467.278825] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1471.176918] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1479.667281] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1481.691539] Lustre: DEBUG MARKER: create test files [ 1513.443134] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1513.450785] Lustre: Skipped 10 previous similar messages [ 1513.459479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1513.463085] Lustre: Skipped 14 previous similar messages [ 1517.507836] Lustre: server umount lustre-OST0000 complete [ 1521.763826] Lustre: server umount lustre-MDT0000 complete [ 1531.967563] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 1534.863775] Key type lgssc unregistered [ 1535.182095] LNet: 56561:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1535.200642] LNetError: 56561:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1535.224139] LNet: Removed LNI 192.168.206.125@tcp [ 1536.097174] Key type .llcrypt unregistered [ 1536.099165] Key type ._llcrypt unregistered [ 1569.567776] Key type ._llcrypt registered [ 1569.574228] Key type .llcrypt registered [ 1569.812411] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 1571.433770] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1571.661039] alg: No test for adler32 (adler32-zlib) [ 1572.935461] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 1573.292350] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 1575.023163] Key type lgssc registered [ 1576.336283] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1584.350094] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1584.383174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1585.848522] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1590.263122] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1598.264310] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1604.476154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1604.827670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1610.737770] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1610.985298] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1611.040272] Lustre: 59295:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1611.047655] Lustre: 59295:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1611.053510] Lustre: 59295:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1611.058503] Lustre: 59295:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1611.064254] Lustre: 59295:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1611.068854] Lustre: 59295:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1617.052406] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1622.007266] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1626.926638] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1635.151545] Lustre: server umount lustre-OST0000 complete [ 1648.341714] Lustre: server umount lustre-MDT0000 complete [ 1660.952866] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 1664.307931] Key type lgssc unregistered [ 1664.583576] LNet: 60902:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1664.599885] LNetError: 60902:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1664.643677] LNet: Removed LNI 192.168.206.125@tcp [ 1665.919603] Key type .llcrypt unregistered [ 1665.926041] Key type ._llcrypt unregistered [ 1698.060556] Key type ._llcrypt registered [ 1698.067668] Key type .llcrypt registered [ 1698.209828] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 1699.456066] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1699.608874] alg: No test for adler32 (adler32-zlib) [ 1700.714368] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 1700.940507] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 1702.671205] Key type lgssc registered [ 1703.885438] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1711.402044] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1711.425257] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1712.943176] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1717.325574] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1724.062640] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1730.252069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1730.693224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1737.531291] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1737.836846] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1737.932309] Lustre: 63737:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1737.947489] Lustre: 63737:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1737.956013] Lustre: 63737:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1737.967945] Lustre: 63737:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1737.981154] Lustre: 63737:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1737.994166] Lustre: 63737:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1746.928825] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1753.057454] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1768.934770] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1768.944281] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1768.971096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1770.456685] Lustre: server umount lustre-OST0000 complete [ 1774.399100] Lustre: server umount lustre-MDT0000 complete [ 1784.238695] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 1786.997896] Key type lgssc unregistered [ 1787.359979] LNet: 65294:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1787.373727] LNetError: 65294:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1787.392363] LNet: Removed LNI 192.168.206.125@tcp [ 1788.286152] Key type .llcrypt unregistered [ 1788.289683] Key type ._llcrypt unregistered [ 1804.541676] Key type ._llcrypt registered [ 1804.544780] Key type .llcrypt registered [ 1804.627352] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 1819.800442] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 1820.738392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1820.870090] alg: No test for adler32 (adler32-zlib) [ 1821.846939] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 1822.238588] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 1823.935228] Key type lgssc registered [ 1825.021851] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1856.844899] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1856.895488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1858.135498] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1858.165446] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1858.250555] Lustre: lustre-MDT0000: new disk, initializing [ 1858.327645] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1858.347974] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1862.017349] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1872.249858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1872.375563] Lustre: 68448:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1872.461142] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1872.465790] Lustre: Skipped 1 previous similar message [ 1872.525102] Lustre: lustre-MDT0001: new disk, initializing [ 1872.646937] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1872.701568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1872.714939] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1877.093371] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1886.233583] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1892.037277] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1900.608632] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1900.982734] Lustre: lustre-OST0000: new disk, initializing [ 1900.987253] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1901.000357] Lustre: 70417:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1901.105950] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1907.230091] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1907.250204] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1907.350769] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1909.389901] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 1918.724442] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1923.903291] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1924.150706] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1928.974994] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1929.182142] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1932.772909] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1932.777072] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1932.789530] Lustre: Skipped 1 previous similar message [ 1937.890440] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1937.894255] Lustre: Skipped 1 previous similar message [ 1943.025889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1943.047434] Lustre: Skipped 1 previous similar message [ 1946.079941] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1946.330224] Lustre: server umount lustre-OST0000 complete [ 1953.253294] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1953.255025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1953.279669] Lustre: Skipped 1 previous similar message [ 1953.296718] Lustre: Skipped 1 previous similar message [ 1956.752542] Lustre: server umount lustre-MDT0000 complete [ 1960.870226] LustreError: 68439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788154250 with bad export cookie 15338379898824794911 [ 1960.879888] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1960.887397] LustreError: 68439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1961.262050] Lustre: server umount lustre-MDT0001 complete [ 1970.859741] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:30:58 (1788154258) [ 1981.843092] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 2023.712818] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 2034.161209] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 2072.129620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2072.527543] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2072.550792] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2072.674669] Lustre: lustre-MDT0000: new disk, initializing [ 2072.787129] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2072.844964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2077.659909] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2089.701476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2089.831675] Lustre: 76447:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2089.933972] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2089.944413] Lustre: Skipped 1 previous similar message [ 2090.071918] Lustre: lustre-MDT0001: new disk, initializing [ 2090.165946] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2090.222853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2090.238813] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2094.779672] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2104.758913] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2110.251281] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2117.474469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2117.855786] Lustre: lustre-OST0000: new disk, initializing [ 2117.862044] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2117.872708] Lustre: 78416:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2118.006147] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2124.183453] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2126.430921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2126.452108] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2126.545548] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2132.998584] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2137.869437] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2138.138166] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2143.132243] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2143.317684] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2147.296512] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2147.308514] LustreError: Skipped 1 previous similar message [ 2147.311480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2147.327119] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2147.344122] Lustre: Skipped 1 previous similar message [ 2151.165919] Lustre: server umount lustre-OST0000 complete [ 2157.026594] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2157.033954] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2157.048987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2157.055120] Lustre: Skipped 1 previous similar message [ 2160.930336] Lustre: server umount lustre-MDT0000 complete [ 2162.655946] LustreError: 76459:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2162.684018] LustreError: 76459:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2164.358109] LustreError: 76439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788154453 with bad export cookie 15338379898824795793 [ 2164.362662] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2164.369226] LustreError: 76439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2164.782247] Lustre: server umount lustre-MDT0001 complete [ 2172.146954] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:34:20 (1788154460) [ 2178.511549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2178.936304] LustreError: 80829:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2179.022766] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2183.013898] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2184.162605] LustreError: 80830:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2184.178078] LustreError: 80830:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2189.283051] LustreError: 80829:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2191.159831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2196.084248] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2202.719266] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2208.496989] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2214.127078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2214.415112] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2214.422775] Lustre: Skipped 1 previous similar message [ 2219.530831] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2225.616879] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2228.619569] Lustre: 83499:0:(genops.c:1803:obd_export_evict_by_uuid()) MGS: evicting 8f0291b3-7e9e-45e5-8b58-16de69b96231 at adminstrative request [ 2229.735756] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2229.748605] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2238.953081] Lustre: MGS: Client 8f0291b3-7e9e-45e5-8b58-16de69b96231 (at 0@lo) reconnecting [ 2238.963562] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xd4dcdf407371aa03 to 0xd4dcdf407371aca3 [ 2238.994771] Lustre: MGC192.168.206.125@tcp: Connection restored to 0@lo (at 0@lo) [ 2267.795773] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2268.044373] Lustre: lustre-OST0001: new disk, initializing [ 2268.048247] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2268.054491] Lustre: 83895:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2273.905963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2273.913537] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2273.992923] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2274.417634] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2282.422843] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2289.638747] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2289.647800] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2289.656333] Lustre: Skipped 2 previous similar messages [ 2289.664478] Lustre: Skipped 2 previous similar messages [ 2291.373713] Lustre: server umount lustre-OST0001 complete [ 2294.761778] LustreError: 82649:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2294.776914] LustreError: 82649:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2305.020993] LustreError: 82648:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2305.044560] LustreError: 82648:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2309.087153] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2309.466849] Lustre: server umount lustre-OST0000 complete [ 2314.210229] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2314.227907] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2314.245241] Lustre: Skipped 2 previous similar messages [ 2320.133806] Lustre: server umount lustre-MDT0000 complete [ 2323.435211] LustreError: 80830:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2323.462484] LustreError: 80830:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2324.518286] LustreError: 80809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788154613 with bad export cookie 15338379898824797347 [ 2324.527382] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2324.546069] LustreError: 80809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2325.121632] Lustre: server umount lustre-MDT0001 complete [ 2334.314455] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:37:02 (1788154622) [ 2336.126503] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2338.419697] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:37:06 (1788154626) [ 2340.049682] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2342.255088] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:37:10 (1788154630) [ 2344.175053] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2346.359914] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:37:14 (1788154634) [ 2348.426471] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2350.418344] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:37:18 (1788154638) [ 2351.971490] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2358.359125] loop: module loaded [ 2358.366446] loop0: detected capacity change from 0 to 4294967296 [ 2358.415511] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2360.354555] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2363.501315] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2367.178382] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2392.309166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2392.354866] LustreError: 88015:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2392.368274] LustreError: 88015:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2392.386391] LustreError: 88015:0:(tgt_mount.c:2656:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2392.406832] LustreError: 88015:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2411.505054] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 2421.860417] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 2455.968833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2456.202556] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2456.220588] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2456.275589] Lustre: lustre-MDT0000: new disk, initializing [ 2456.318822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2456.324343] Lustre: Skipped 1 previous similar message [ 2456.334206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2461.107595] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2471.779521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2471.982799] Lustre: 90880:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2472.519834] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2477.396871] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2487.551380] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2493.591709] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2500.392479] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2500.706729] Lustre: 92849:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2507.796927] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2509.378516] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2509.486080] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2516.647530] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2521.479256] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2521.778189] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2526.927546] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2527.224379] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2530.274677] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2530.287287] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2530.288852] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2530.305398] Lustre: Skipped 2 previous similar messages [ 2530.325632] Lustre: Skipped 12 previous similar messages [ 2535.227336] Lustre: server umount lustre-OST0000 complete [ 2545.637373] LustreError: 90891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2545.764174] Lustre: server umount lustre-MDT0000 complete [ 2549.239813] LustreError: 90871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788154838 with bad export cookie 15338379898824797963 [ 2549.244693] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2549.259493] LustreError: 90871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2549.624423] Lustre: server umount lustre-MDT0001 complete [ 2556.693333] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:40:45 (1788154845) [ 2562.267956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2566.582355] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2574.958335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2579.404954] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2587.527572] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2592.923838] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2599.126293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2599.441492] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2599.445721] Lustre: Skipped 4 previous similar messages [ 2605.836704] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2612.608020] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2617.243720] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2617.393186] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2621.414603] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2621.625832] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2628.170224] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2628.301972] Lustre: lustre-OST0001: new disk, initializing [ 2628.304703] Lustre: Skipped 2 previous similar messages [ 2628.308122] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2628.310835] Lustre: Skipped 3 previous similar messages [ 2628.317485] Lustre: 98634:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2634.293144] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2634.868712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2634.892623] Lustre: Skipped 2 previous similar messages [ 2634.907903] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2635.022747] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2643.614356] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2648.551216] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2648.839057] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2653.510175] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2653.844378] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2674.092932] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2696.746674] Lustre: Failing over lustre-MDT0000 [ 2698.082458] Lustre: lustre-MDT0000: Not available for connect from 192.168.206.25@tcp (stopping) [ 2698.093141] Lustre: Skipped 3 previous similar messages [ 2698.220075] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2698.235319] LustreError: Skipped 1 previous similar message [ 2698.243285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2698.263690] Lustre: Skipped 2 previous similar messages [ 2699.398914] Lustre: server umount lustre-MDT0000 complete [ 2701.797451] LustreError: 95993:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2701.821703] LustreError: 95993:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 2717.157407] Lustre: 66670:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788154990/real 1788154990] req@00000000c135b4c0 x1875015485368704/t0(0) o400->MGC192.168.206.125@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788155006 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2717.191207] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2724.507166] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2724.509071] LDISKFS-fs (dm-0): recovery complete [ 2724.518674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2728.426130] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xd4dcdf4073727717 [ 2728.432374] Lustre: MGC192.168.206.125@tcp: Connection restored to 0@lo (at 0@lo) [ 2728.805532] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2733.049397] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2733.885689] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2733.887461] Lustre: 99252:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2734.072507] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2734.166329] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2734.489159] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2734.507389] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2734.521882] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 2734.852097] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2734.864792] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2735.183154] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2735.188644] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2735.524460] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2735.537500] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 2 previous similar messages [ 2735.872312] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2735.893607] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2736.215153] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2736.226504] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2737.537123] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2737.567465] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 2737.896663] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2737.907300] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 2738.231136] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2738.241285] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 2741.736508] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2741.747469] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 2742.067956] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2742.079291] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2742.407124] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2742.425211] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2749.745931] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2749.763187] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 2750.080156] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2750.096331] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2750.415115] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2750.422343] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2765.919859] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2765.928887] Lustre: 101489:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 2766.257540] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2766.273578] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2766.591303] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2766.596997] LustreError: 101489:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2793.143455] Lustre: 101489:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2793.172329] Lustre: 101489:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2793.183217] Lustre: 101489:0:(genops.c:1630:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 75e8500c-a4e4-40c3-8817-da0abc59e79a@192.168.206.25@tcp [ 2793.203415] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2793.210943] Lustre: 101489:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000bca9cac4 x1875015470488960/t0(8589934782) o35->75e8500c-a4e4-40c3-8817-da0abc59e79a@192.168.206.25@tcp:258/0 lens 392/0 e 0 to 0 dl 1788155093 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2793.249212] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2793.262924] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2793.269545] Lustre: lustre-MDT0000: Denying connection for new client 75e8500c-a4e4-40c3-8817-da0abc59e79a (at 192.168.206.25@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2793.314374] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2793.321080] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2793.322198] Lustre: Skipped 2 previous similar messages [ 2793.384147] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2793.388149] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 2809.323152] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2809.336072] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2809.352499] Lustre: Skipped 3 previous similar messages [ 2813.451047] Lustre: server umount lustre-OST0000 complete [ 2826.211777] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2826.219423] Lustre: Skipped 9 previous similar messages [ 2831.339055] LustreError: 97086:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2831.353250] LustreError: 97086:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 41 previous similar messages [ 2832.352032] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2832.595784] Lustre: server umount lustre-OST0001 complete [ 2842.516424] Lustre: server umount lustre-MDT0000 complete [ 2846.184808] LustreError: 95245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788155135 with bad export cookie 15338379898824849175 [ 2846.187510] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2846.194824] LustreError: 95245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2846.521241] Lustre: server umount lustre-MDT0001 complete [ 2854.124478] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:45:42 (1788155142) [ 2861.186888] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2861.197349] Lustre: *** cfs_fail_loc=197, val=0*** [ 2876.776596] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2881.889129] LustreError: 103463:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.125@tcp: failed processing log, type 4: rc = -110 [ 2887.039657] Lustre: *** cfs_fail_loc=197, val=0*** [ 2887.050469] Lustre: Skipped 65 previous similar messages [ 2907.616112] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2907.625424] Lustre: Skipped 2 previous similar messages [ 2913.564160] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2920.826165] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2922.774651] Lustre: server umount lustre-OST0000 complete [ 2930.578304] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:46:58 (1788155218) [ 2952.308967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2957.709102] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 2967.654113] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2971.919250] LustreError: 105114:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2971.937029] LustreError: 105114:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2971.942673] LustreError: 105114:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 2972.326895] Lustre: server umount lustre-MDT0000 complete [ 3006.447956] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 3016.443920] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 3055.293355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3055.538788] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3055.578576] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3055.698400] Lustre: lustre-MDT0000: new disk, initializing [ 3055.813079] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3060.467346] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3071.595634] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3071.729650] Lustre: 109291:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3071.973607] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3076.780319] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3086.602629] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3092.157683] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3099.586866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3100.029951] Lustre: 111258:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3106.350201] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3106.497626] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3115.308750] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3121.125727] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3121.405376] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3126.026500] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3126.296630] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3131.880115] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3131.897664] LustreError: Skipped 3 previous similar messages [ 3131.908063] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3131.926589] Lustre: Skipped 5 previous similar messages [ 3131.930426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3131.940207] Lustre: Skipped 6 previous similar messages [ 3134.601614] Lustre: server umount lustre-OST0000 complete [ 3144.866057] Lustre: server umount lustre-MDT0000 complete [ 3147.744451] LustreError: 109300:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3147.766145] LustreError: 109300:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3148.873896] LustreError: 109282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788155438 with bad export cookie 15338379898824859906 [ 3148.883277] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3148.888876] LustreError: 109282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3149.303746] Lustre: server umount lustre-MDT0001 complete [ 3159.138756] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:50:47 (1788155447) [ 3167.912558] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 3170.392951] Key type lgssc unregistered [ 3170.681548] LNet: 113746:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3170.690557] LNetError: 113746:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3170.719578] LNet: Removed LNI 192.168.206.125@tcp [ 3171.504491] Key type .llcrypt unregistered [ 3171.506610] Key type ._llcrypt unregistered [ 3188.539657] Key type ._llcrypt registered [ 3188.542422] Key type .llcrypt registered [ 3188.728844] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 3204.704343] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 3206.315879] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3206.585423] alg: No test for adler32 (adler32-zlib) [ 3207.878161] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 3208.163459] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 3209.863176] Key type lgssc registered [ 3211.232956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3254.917824] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3254.967146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3256.164433] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3256.185690] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3256.256941] Lustre: lustre-MDT0000: new disk, initializing [ 3256.327854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3256.365405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3260.834459] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3271.599302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3272.041348] Lustre: lustre-OST0000: new disk, initializing [ 3272.049783] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3272.057580] Lustre: Skipped 1 previous similar message [ 3272.067725] Lustre: 117921:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3272.228953] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3279.189373] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3281.050934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3281.062762] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3281.114463] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3291.481647] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3291.747441] Lustre: lustre-OST0001: new disk, initializing [ 3291.753546] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3291.759399] Lustre: 118785:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3291.903835] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3299.629023] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3301.122084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3301.131297] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3301.190301] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3311.588461] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3311.600487] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3311.604374] Lustre: Skipped 1 previous similar message [ 3311.643435] Lustre: Skipped 1 previous similar message [ 3315.490160] Lustre: server umount lustre-MDT0000 complete [ 3321.716303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3326.722794] LustreError: 117110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788155615 with bad export cookie 17724348055617071213 [ 3326.733729] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3326.938352] Lustre: server umount lustre-OST0000 complete [ 3330.614426] Lustre: server umount lustre-OST0001 complete [ 3338.088917] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:53:46 (1788155626) [ 3356.215506] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:54:04 (1788155644) [ 3365.653856] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 3390.743487] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:54:38 (1788155678) [ 3425.396139] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 3434.652785] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 3472.342745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3472.566909] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3472.613997] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3472.705811] Lustre: lustre-MDT0000: new disk, initializing [ 3472.769743] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3472.786363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3477.687891] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3487.608159] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3487.707385] Lustre: 126537:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3487.803904] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3487.808244] Lustre: Skipped 1 previous similar message [ 3487.922255] Lustre: lustre-MDT0001: new disk, initializing [ 3488.018529] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3488.061835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3488.074940] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3492.399211] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3501.483325] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3505.975728] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3511.308966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3511.522152] Lustre: lustre-OST0000: new disk, initializing [ 3511.525478] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3511.533529] Lustre: 128506:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3511.617344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3517.245477] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3517.514755] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3517.528366] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3517.626833] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3526.698151] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3531.099611] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3531.279656] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3535.760416] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3535.967720] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3538.410355] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3538.411769] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3538.415595] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3538.421376] LustreError: Skipped 1 previous similar message [ 3538.439435] Lustre: Skipped 1 previous similar message [ 3543.532965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3543.540659] Lustre: Skipped 2 previous similar messages [ 3548.655637] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3548.669484] Lustre: Skipped 1 previous similar message [ 3552.223258] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3552.447695] Lustre: server umount lustre-OST0000 complete [ 3558.885849] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3558.890941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3558.902672] Lustre: Skipped 1 previous similar message [ 3558.925267] Lustre: Skipped 1 previous similar message [ 3567.011337] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3567.018065] Lustre: Skipped 3 previous similar messages [ 3571.167140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3571.392268] Lustre: server umount lustre-MDT0000 complete [ 3575.165259] LustreError: 126528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788155864 with bad export cookie 17724348055617072438 [ 3575.167893] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3575.177267] LustreError: 126528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3575.587927] Lustre: server umount lustre-MDT0001 complete [ 3586.394654] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 01:57:54 (1788155874) [ 3594.016556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3594.698169] LustreError: 130921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3594.727878] LustreError: 130921:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3594.812726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3599.341753] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3599.846559] LustreError: 130922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3603.941896] LustreError: 130921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3609.056854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3609.075913] LustreError: 130922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3616.045538] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3626.756530] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3633.203976] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3640.848195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3641.347244] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3641.353324] Lustre: Skipped 1 previous similar message [ 3648.856117] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3658.386851] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3673.822533] Lustre: *** cfs_fail_loc=159, val=0*** [ 3690.419040] Lustre: lustre-MDT0000: Client c5460d9d-e9b5-41d7-8535-010b8e4d1408 (at 192.168.206.25@tcp) reconnecting [ 3692.262070] Lustre: *** cfs_fail_loc=159, val=0*** [ 3692.264242] Lustre: Skipped 5 previous similar messages [ 3707.777629] Lustre: lustre-MDT0000: Client c5460d9d-e9b5-41d7-8535-010b8e4d1408 (at 192.168.206.25@tcp) reconnecting [ 3711.971366] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3711.982310] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3711.999896] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3716.064205] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3716.078692] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3717.757719] Lustre: server umount lustre-OST0000 complete [ 3722.211141] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3726.306458] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3726.316064] Lustre: Skipped 1 previous similar message [ 3728.157674] Lustre: server umount lustre-MDT0000 complete [ 3731.424751] LustreError: 130921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3731.448788] LustreError: 130921:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3732.032880] LustreError: 130903:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788156021 with bad export cookie 17724348055617073320 [ 3732.042507] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3732.050099] LustreError: 130903:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3732.756563] Lustre: server umount lustre-MDT0001 complete [ 3744.212885] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 3747.289292] Key type lgssc unregistered [ 3747.569900] LNet: 134721:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3747.580795] LNetError: 134721:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3747.610756] LNet: Removed LNI 192.168.206.125@tcp [ 3748.551671] Key type .llcrypt unregistered [ 3748.555320] Key type ._llcrypt unregistered [ 3772.088383] Key type ._llcrypt registered [ 3772.097917] Key type .llcrypt registered [ 3772.397710] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 3774.225916] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3774.282392] alg: No test for adler32 (adler32-zlib) [ 3775.372912] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 3775.676131] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 3777.391142] Key type lgssc registered [ 3778.657885] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3786.439174] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3786.481046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3787.894589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3793.397670] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3798.499254] LustreError: 136148:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3798.521266] LustreError: 136148:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3803.140350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3803.552059] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3808.241633] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3815.824744] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3821.836738] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3829.782926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3830.291231] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3833.411249] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3838.314855] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 3847.812738] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3857.453576] Lustre: *** cfs_fail_loc=159, val=0*** [ 3873.644991] Lustre: lustre-MDT0000: Client fb3c9c00-0af6-4a43-93b7-4658d420e19a (at 192.168.206.25@tcp) reconnecting [ 3876.165281] Lustre: *** cfs_fail_loc=159, val=0*** [ 3876.171750] Lustre: Skipped 2 previous similar messages [ 3891.588988] Lustre: lustre-MDT0001: Client fb3c9c00-0af6-4a43-93b7-4658d420e19a (at 192.168.206.25@tcp) reconnecting [ 3893.137306] Lustre: *** cfs_fail_loc=159, val=0*** [ 3893.140419] Lustre: Skipped 1 previous similar message [ 3908.472909] Lustre: lustre-MDT0001: Client fb3c9c00-0af6-4a43-93b7-4658d420e19a (at 192.168.206.25@tcp) reconnecting [ 3913.380672] Lustre: 139191:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3915.975446] Lustre: *** cfs_fail_loc=159, val=0*** [ 3915.979850] Lustre: Skipped 4 previous similar messages [ 3931.513116] Lustre: lustre-MDT0000: Client a3285dec-dcf3-42f6-a187-5de0bed028af (at 192.168.206.25@tcp) reconnecting [ 3933.613606] Lustre: *** cfs_fail_loc=159, val=0*** [ 3933.618160] Lustre: Skipped 13 previous similar messages [ 3949.988257] Lustre: lustre-MDT0000: Client a3285dec-dcf3-42f6-a187-5de0bed028af (at 192.168.206.25@tcp) reconnecting [ 3951.576985] Lustre: 139485:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3971.558121] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3971.568241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3971.573160] LustreError: Skipped 1 previous similar message [ 3971.580524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3971.598883] Lustre: Skipped 1 previous similar message [ 3971.622746] Lustre: Skipped 1 previous similar message [ 3976.538157] Lustre: server umount lustre-OST0000 complete [ 3981.799243] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3981.809281] Lustre: Skipped 1 previous similar message [ 3981.810290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3986.536568] Lustre: server umount lustre-MDT0000 complete [ 3986.916648] LustreError: 136145:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3986.946137] LustreError: 136145:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3990.557846] LustreError: 136128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788156279 with bad export cookie 3866244797697280450 [ 3990.562850] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3990.574089] LustreError: 136128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3990.782352] Lustre: server umount lustre-MDT0001 complete [ 4001.576647] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 4004.465305] Key type lgssc unregistered [ 4004.821834] LNet: 140563:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4004.841670] LNetError: 140563:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4005.932339] LNet: Removed LNI 192.168.206.125@tcp [ 4006.887966] Key type .llcrypt unregistered [ 4006.890945] Key type ._llcrypt unregistered [ 4032.328472] Key type ._llcrypt registered [ 4032.334071] Key type .llcrypt registered [ 4032.455751] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 4033.277592] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4033.389135] alg: No test for adler32 (adler32-zlib) [ 4034.574259] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 4034.923288] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 4036.639182] Key type lgssc registered [ 4037.787345] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4045.353740] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4045.368680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4046.751892] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4051.013477] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4056.034780] LustreError: 141978:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4056.062841] LustreError: 141978:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4059.431632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4059.841762] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4064.424439] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4071.510314] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4076.276649] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4082.543335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4082.949831] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4083.954732] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4088.898982] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4091.124925] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4096.548073] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4102.154545] Lustre: 144556:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4104.942272] Lustre: 144604:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4106.657647] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4107.526282] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4111.762035] Lustre: 144753:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4112.819256] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4114.046216] Lustre: 144862:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4120.033859] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4120.038984] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4120.052079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4122.099648] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4122.112279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4122.112897] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4122.141958] Lustre: Skipped 1 previous similar message [ 4124.652933] Lustre: server umount lustre-OST0000 complete [ 4131.808346] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4131.822666] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4131.843082] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4135.341692] Lustre: server umount lustre-MDT0000 complete [ 4137.440942] LustreError: 141978:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4137.456705] LustreError: 141978:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4138.931611] LustreError: 141960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788156428 with bad export cookie 10541493916380374732 [ 4138.933425] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4138.947235] LustreError: 141960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4139.435747] Lustre: server umount lustre-MDT0001 complete [ 4153.773122] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 4156.983758] Key type lgssc unregistered [ 4157.234882] LNet: 145915:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4157.243635] LNetError: 145915:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4157.271606] LNet: Removed LNI 192.168.206.125@tcp [ 4158.176148] Key type .llcrypt unregistered [ 4158.179123] Key type ._llcrypt unregistered [ 4181.094822] Key type ._llcrypt registered [ 4181.098687] Key type .llcrypt registered [ 4181.195378] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 4182.036621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4182.069538] alg: No test for adler32 (adler32-zlib) [ 4183.188687] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 4183.448549] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 4185.112130] Key type lgssc registered [ 4186.102994] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4193.062553] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4193.081146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4194.440217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4198.604331] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4204.016217] LustreError: 147329:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4204.030259] LustreError: 147329:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4206.298861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4206.637237] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4210.195646] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4216.410339] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4222.035427] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4228.808452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4229.156558] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4231.230683] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4236.072547] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4239.425798] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4242.941920] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4249.895881] Lustre: *** cfs_fail_loc=159, val=0*** [ 4266.349543] Lustre: lustre-MDT0000: Client d596b4e7-db96-4ad8-ab76-272e6ae943f2 (at 192.168.206.25@tcp) reconnecting [ 4272.097978] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4272.108696] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4272.118904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4272.128016] Lustre: Skipped 1 previous similar message [ 4272.141564] Lustre: Skipped 1 previous similar message [ 4273.256882] Lustre: server umount lustre-OST0000 complete [ 4278.242486] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4278.254519] LustreError: Skipped 8 previous similar messages [ 4278.257034] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4278.269771] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4280.290195] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4280.294259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4280.317944] Lustre: Skipped 1 previous similar message [ 4283.050165] Lustre: server umount lustre-MDT0000 complete [ 4285.408333] LustreError: 147329:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4285.428142] LustreError: 147329:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4286.323385] LustreError: 147311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788156575 with bad export cookie 3439622152254750670 [ 4286.330644] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4286.332431] LustreError: 147311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4286.685422] Lustre: server umount lustre-MDT0001 complete [ 4296.834100] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 4299.569613] Key type lgssc unregistered [ 4299.845210] LNet: 150990:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4299.853306] LNetError: 150990:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4299.866474] LNet: Removed LNI 192.168.206.125@tcp [ 4300.643981] Key type .llcrypt unregistered [ 4300.647133] Key type ._llcrypt unregistered [ 4322.838978] Key type ._llcrypt registered [ 4322.841270] Key type .llcrypt registered [ 4322.946543] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 4324.125864] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4324.167305] alg: No test for adler32 (adler32-zlib) [ 4325.170594] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 4325.371379] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 4327.007271] Key type lgssc registered [ 4328.097243] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4335.609517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4335.622832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4337.008565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4340.400975] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4347.370528] LustreError: 152399:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4347.408533] LustreError: 152399:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4349.113934] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4349.509864] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4354.399570] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4361.567353] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4367.639851] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4375.048782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4375.451971] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4381.206962] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4382.643227] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4389.968831] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4393.936566] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4415.448750] Lustre: lustre-MDT0001: evicting 6b3a7efe-3bb8-4fdf-8185-7d1cdbdef8d5 (at 192.168.206.25@tcp) by administrative request [ 4415.467254] Lustre: Skipped 1 previous similar message [ 4444.128550] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4444.138922] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4444.163222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4445.155917] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4445.160270] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4449.399463] Lustre: server umount lustre-OST0000 complete [ 4455.396155] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4455.404888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4455.418385] Lustre: Skipped 1 previous similar message [ 4455.435222] Lustre: Skipped 2 previous similar messages [ 4459.208464] Lustre: server umount lustre-MDT0000 complete [ 4460.520641] LustreError: 152399:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4460.545971] LustreError: 152399:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4462.673051] LustreError: 152378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788156751 with bad export cookie 3371915155133786012 [ 4462.678357] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4462.683317] LustreError: 152378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4463.082580] Lustre: server umount lustre-MDT0001 complete [ 4474.464818] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 4477.101660] Key type lgssc unregistered [ 4477.361200] LNet: 156470:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4477.371045] LNetError: 156470:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4477.386368] LNet: Removed LNI 192.168.206.125@tcp [ 4478.199468] Key type .llcrypt unregistered [ 4478.208787] Key type ._llcrypt unregistered [ 4550.281290] Key type ._llcrypt registered [ 4550.285737] Key type .llcrypt registered [ 4550.445580] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 4551.514267] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4551.559724] alg: No test for adler32 (adler32-zlib) [ 4552.646728] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 4552.977409] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 4554.695966] Key type lgssc registered [ 4556.178662] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4564.885928] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4564.920524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4566.476687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4571.315721] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4576.739118] LustreError: 158959:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4576.762440] LustreError: 158959:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4580.971536] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4581.432328] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4586.933756] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4595.059589] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4601.246411] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4607.628390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4607.961141] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4611.058099] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4614.077951] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4620.978886] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4626.423963] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4632.209135] Lustre: server umount lustre-OST0000 complete [ 4636.641596] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4636.646990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4636.656089] Lustre: Skipped 1 previous similar message [ 4641.772345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4641.778239] Lustre: Skipped 2 previous similar messages [ 4645.004325] Lustre: server umount lustre-MDT0000 complete [ 4646.880588] LustreError: 158959:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4646.928126] LustreError: 158959:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4648.939081] LustreError: 158939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788156938 with bad export cookie 3556351374646046642 [ 4648.942812] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4648.953866] LustreError: 158939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4659.309775] Lustre: server umount lustre-MDT0001 complete [ 4668.730542] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 4671.067556] Key type lgssc unregistered [ 4671.271945] LNet: 162501:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4671.281446] LNetError: 162501:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4671.298167] LNet: Removed LNI 192.168.206.125@tcp [ 4671.986343] Key type .llcrypt unregistered [ 4671.989290] Key type ._llcrypt unregistered [ 4696.163213] Key type ._llcrypt registered [ 4696.167466] Key type .llcrypt registered [ 4696.295402] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 4697.232673] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4697.280880] alg: No test for adler32 (adler32-zlib) [ 4698.492789] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 4698.648717] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 4700.271157] Key type lgssc registered [ 4702.077099] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4719.103509] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:16:47 (1788157007) [ 4734.956624] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 4743.685567] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 4776.051872] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4776.084556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4777.385395] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4777.428454] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4777.519926] Lustre: lustre-MDT0000: new disk, initializing [ 4777.676598] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4777.700692] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4782.612377] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4792.260782] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4792.382883] Lustre: 166762:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4792.422095] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4792.430199] Lustre: Skipped 1 previous similar message [ 4792.505720] Lustre: lustre-MDT0001: new disk, initializing [ 4792.559800] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4792.596713] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4792.606111] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4796.844274] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4806.908537] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4812.883671] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4821.813217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4822.013583] Lustre: lustre-OST0000: new disk, initializing [ 4822.017963] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4822.024430] Lustre: 168731:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4822.134614] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4829.325393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4829.339205] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4829.565738] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4830.362553] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4841.183046] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4855.264121] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4855.270852] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4855.273421] LustreError: Skipped 1 previous similar message [ 4855.274189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4855.334811] Lustre: Skipped 1 previous similar message [ 4856.592709] Lustre: server umount lustre-OST0000 complete [ 4864.480277] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4864.495687] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4864.518951] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4864.521530] Lustre: Skipped 1 previous similar message [ 4865.508519] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4865.537042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4865.545498] Lustre: Skipped 1 previous similar message [ 4866.996993] Lustre: server umount lustre-MDT0000 complete [ 4870.666596] LustreError: 166757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788157159 with bad export cookie 14687991307986506278 [ 4870.668561] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4870.681176] LustreError: 166757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4871.048558] Lustre: server umount lustre-MDT0001 complete [ 4883.104865] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 4886.125567] Key type lgssc unregistered [ 4886.447126] LNet: 170636:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4886.454842] LNetError: 170636:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4886.471165] LNet: Removed LNI 192.168.206.125@tcp [ 4887.435295] Key type .llcrypt unregistered [ 4887.437039] Key type ._llcrypt unregistered [ 4909.363385] Key type ._llcrypt registered [ 4909.367476] Key type .llcrypt registered [ 4909.505739] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 4910.578539] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4910.714271] alg: No test for adler32 (adler32-zlib) [ 4911.815262] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 4912.100032] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 4913.831247] Key type lgssc registered [ 4914.793925] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4922.277329] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4922.318809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4923.886419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4928.050137] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4934.112464] LustreError: 172050:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4934.149668] LustreError: 172050:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4939.048233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4939.239653] LustreError: 172051:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4939.680307] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4946.453088] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4955.989436] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4962.511296] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4968.339857] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4968.662046] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4969.723856] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4974.043234] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 4980.647997] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4990.605118] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4990.630497] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4990.646345] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4990.652272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.664699] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4991.766218] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4991.794534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4991.827268] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4991.837060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.851935] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4992.871782] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4992.898178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4992.912905] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4992.921144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.936478] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4993.760193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.897989] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4994.907322] Lustre: Skipped 1 previous similar message [ 4994.943573] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4994.958758] Lustre: Skipped 1 previous similar message [ 4994.973861] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4994.979409] Lustre: Skipped 1 previous similar message [ 4994.988192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.000400] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4995.010658] Lustre: Skipped 1 previous similar message [ 4996.202232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.144584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.225224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.216440] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4999.228053] Lustre: Skipped 3 previous similar messages [ 4999.257058] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4999.271444] Lustre: Skipped 3 previous similar messages [ 4999.282681] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4999.293980] Lustre: Skipped 3 previous similar messages [ 4999.296701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.308097] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4999.311878] Lustre: Skipped 3 previous similar messages [ 5000.524777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.596241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.647604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.573578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.412187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.168621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.961867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.925323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.838556] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5007.848035] Lustre: Skipped 8 previous similar messages [ 5007.862507] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5007.869995] Lustre: Skipped 8 previous similar messages [ 5007.876448] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5007.881245] Lustre: Skipped 8 previous similar messages [ 5007.887023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.894104] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5007.897497] Lustre: Skipped 8 previous similar messages [ 5008.709138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.535308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.543805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.435653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.236262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.106075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.944430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.768072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.691925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.577216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.324949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.192639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.104287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.032840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.996777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.002159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.112144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.853273] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5023.858450] Lustre: Skipped 17 previous similar messages [ 5023.868636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5023.876852] Lustre: Skipped 17 previous similar messages [ 5023.880920] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5023.885569] Lustre: Skipped 17 previous similar messages [ 5023.888860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.897435] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5023.901877] Lustre: Skipped 17 previous similar messages [ 5024.803616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.939864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.991485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.119961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.324170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.479597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.443938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.434418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.325047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.166385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.031121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.053495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.960461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.855277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.688040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.547321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.387390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.262760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.173843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.082991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.881760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.802736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.674623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.702221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.941798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.944750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.948051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.858431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.982893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.954564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.895501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.149500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.060461] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5056.065844] Lustre: Skipped 32 previous similar messages [ 5056.090456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5056.113794] Lustre: Skipped 32 previous similar messages [ 5056.118702] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5056.127140] Lustre: Skipped 32 previous similar messages [ 5056.137501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.161382] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5056.170886] Lustre: Skipped 32 previous similar messages [ 5057.543464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.869774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.843350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.833850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.036954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.928157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.054766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.895703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.105292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.202975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.239724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.162413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.009906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.979890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.643939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.838637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.913606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.788660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.654164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.285898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.465392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.361851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.236196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.928619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.970873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.774380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.822928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.725762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.810706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.050972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.182583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.167578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.036509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.899164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.777687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.506563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.414302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.137327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.988637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.757912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.614297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.521652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.418759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.216278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.066719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.943123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.820917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.663875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.815646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.976129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.242598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.166478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.305456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.217544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.222956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.139647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.082500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.828463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.818043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.564248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.531626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.356733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.207043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.023406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.949328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.865873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.690879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.694137] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5120.700356] Lustre: Skipped 67 previous similar messages [ 5120.730312] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5120.744595] Lustre: Skipped 67 previous similar messages [ 5120.752659] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5120.762598] Lustre: Skipped 67 previous similar messages [ 5120.766366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.782990] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5120.791152] Lustre: Skipped 67 previous similar messages [ 5121.543441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.410911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.248986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.193109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.060105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.046821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.796188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.784091] LustreError: 172070:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5127.795565] LustreError: 172070:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5127.808434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.887544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.886985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.701208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.695218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.630782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.676239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.627581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.697322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.717981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.635162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.454830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.357753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.124205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.090419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.938911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.891657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.766335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.580418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.309141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.176129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.998550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.007764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.872128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.860719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.814354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.796708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.776837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.779954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.778051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.755709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.602578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.521298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.437755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.496991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.361537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.236236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.181960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.015204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.820610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.666661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.566653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.507557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.307686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.192361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.141552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.316524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.602223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.550582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.808639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.035795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.003888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.269442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.059903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.918349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.072264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.601573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.422770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.529719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.833686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.289822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.745764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.287296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.190864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.471218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.549338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.872118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.931259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.204227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.295483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.157422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.224571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.065957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.126211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.924863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.148574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.229278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.047881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.862220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.844987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.787378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.742685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.617196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.618967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.804980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.766461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.830151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.789671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.874528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.872394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.911809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.019671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.015643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.033155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.967647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.877393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.800199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.917934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.920530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.980451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.008563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.997486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.098965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.328952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.460093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.803149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.145230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.239932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.119533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.152073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.389416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.440538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.426319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.286625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.273792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.374513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.453345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.174077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.122019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.045902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.955274] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5248.963364] Lustre: Skipped 127 previous similar messages [ 5248.988695] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5248.995498] Lustre: Skipped 127 previous similar messages [ 5248.999530] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5249.004132] Lustre: Skipped 127 previous similar messages [ 5249.007365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.016968] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5249.022519] Lustre: Skipped 127 previous similar messages [ 5249.919544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.953128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.972574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.074561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.976393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.056698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.810452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.675052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.749648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.794463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.669083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.665785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.749869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.459823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.481206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.567342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.456967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.289768] LustreError: 172070:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5266.296052] LustreError: 172070:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5266.299271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.118729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.091680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.865171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.784707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.634396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.746929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.622651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.453983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.509070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.403980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.358776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.227689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.207814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.133160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.238262] LustreError: 172070:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5280.249576] LustreError: 172070:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5280.280077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.340623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.307540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.461284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.792077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.987485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.924598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.954308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.311550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.320058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.433608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.650071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.629359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.810926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.766718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.000743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.454531] LustreError: 172070:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5298.462976] LustreError: 172070:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5298.475670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.715568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.864263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.105286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.047117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.003483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.089189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.305550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.191571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.117413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.920782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.933948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.068309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.863624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.785221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.699835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.818986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.705686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.674888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.687459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.782326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.675693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.605810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.731895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.870607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.846595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.747878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.711567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.647576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.521157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.472483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.357755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.279548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.395720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.565391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.761921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.938850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.125112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.926699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.869483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.616653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.606987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.369811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.192059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.043123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.975671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.782530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.636476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.602591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.447663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.454406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.479679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.668445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.601474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.481037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.408954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.351881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.278533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.264488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.250950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.226654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.078468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.130928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.185271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.365815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.486206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.356994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.626469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.787829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.633750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.555629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.454385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.683334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.771991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.760355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.569718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.358852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.078851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.178732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.047611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.991511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.983121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.052165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.264074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.039077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.078734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.106423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.060329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.924239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.774219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.607272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.597056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.458504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.387977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.506938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.523056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.442872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.471330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.367067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.203057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.137460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.023903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.940458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.880823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.915090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.884147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.915212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.814142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.784356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.576660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.560514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.433567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.283750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.136130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.938312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.754279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.693285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.451582] LustreError: 172070:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5411.462159] LustreError: 172070:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5411.463614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.359865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.336199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.200722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.124708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.968530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.093255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.245656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.477363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.637142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.783648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.683226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.580853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.479176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.421684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.242240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.239319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.019100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.833316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.710865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.621276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.358690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.302550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.187397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.063131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.834064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.719858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.561753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.477502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.307541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.227927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.137756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.322041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.293390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.256562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.049902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.111449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.989064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.847367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.690709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.771556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.810165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.701104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.861571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.748712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.977892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.083621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.309580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.343973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.476248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.561372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.525476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.571324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.598116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.765145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.784857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.277754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.239700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.418802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.736897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.891772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.909031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.994492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.189859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.525781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.264912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.238350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.396857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.229246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.116771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.223806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.316904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.380868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.371774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.405568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.467655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.701883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.483678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.353694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.427939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.629841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.906662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.673571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.690417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.606080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.543673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.438750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.396762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.369508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.329367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.312775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.388442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.320165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.165641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.106591] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5505.117966] Lustre: Skipped 259 previous similar messages [ 5505.150479] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5505.166311] Lustre: Skipped 259 previous similar messages [ 5505.181543] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5505.196018] Lustre: Skipped 259 previous similar messages [ 5505.214769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.239475] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5505.242919] Lustre: Skipped 259 previous similar messages [ 5506.200144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.992495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.958702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.862929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.729892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.692188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.663932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.484842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.235856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.238045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.234137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.186635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.064798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.935255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.869267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.844245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.720886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.575623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.547788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.629242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.649405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.889048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.018777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.130758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.133445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.040734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.959479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.900949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.865685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.825429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.738979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.624917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.505903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.414899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.435180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.703469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.029391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.000356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.962586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.789890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.813408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.684572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.609921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.485860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.660648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.542567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.506374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.384380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.204932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.000760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.015000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.886906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.117811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.084590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.175200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.992373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.079120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.092264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.938426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.673791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.639452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.603390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.658499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.683648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.526830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.451260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.390802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.341741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.263574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.076650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.012306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.912101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.912246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.835150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.705905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.541602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.395368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.192194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.126356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.091505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.988364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.885520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.730674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.602026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.399547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.148991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.944576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.716292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.702055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.667736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.669294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.616816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.614656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.630358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.648279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.766781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.012386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.152775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.344180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.035292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.816738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.626853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.640338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.831204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.035497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.024487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.106726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.335257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.432264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.475040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.771221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.724806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.901555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.843464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.951645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.216204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.204979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.335527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.243295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.287303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.413463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.609224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.875497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.975153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.861629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.835093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.726463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.797035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.672650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.649431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.477508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.589451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.634177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.531832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.517974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.776602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.090064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.196400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.260821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.366798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.540078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.662962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.448708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.548422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.502278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.900140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.036703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.025475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.189900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.152851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.295242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.441466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.386960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.375196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.415729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.418956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.594600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.519286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.652220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.420148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.452844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.270606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.245378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.285257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.095384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.126697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.888552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.667926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.550893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.646180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.838844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.848585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.855059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.759806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.708689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.614838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.682946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.484729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.551861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.437847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.500839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.319466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.416930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.358371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.245767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.219190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.214915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.171917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.182232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.158513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.251884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.092267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.939529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.864400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.078900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.273433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.715836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.822862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.880398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.798851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.756771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.677528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.626539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.644758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.497816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.520341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.229813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.019640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.905125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.902747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.794404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.658217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.700087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.603578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.672049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.505084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.355973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.082278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.935958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.930824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.852649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.630541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.547521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.355072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.169127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.195070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.079785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.928739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.867812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.706474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.566882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.330144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.280567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.137519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.953624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.811929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.701957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.522074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.372487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.158443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.976727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.091722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.991333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.836112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.598365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.412184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.326555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.339709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.012676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.910789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.625819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.382637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.176788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.901895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.690564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.520491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.479926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.225058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.044643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.025273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.067255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.073839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.995670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.670979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.442681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.337490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.063725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.800221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.571629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.338713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.182926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.950930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.776702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.677142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.493489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.384884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.201612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.164836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.118721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.016443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.907753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.815844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.722497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.662801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.552436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.403645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.249304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.049278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.997332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.942476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.864045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.764091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.702337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.754962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.634313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.585791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.393558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.268711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.203379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.251815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.114259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.022416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.927595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.896565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.622961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.536862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.495832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.363420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.380198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.390912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.318781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.389171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.313073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.428861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.460361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.512099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.699188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.538224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.467778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.422637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.501136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.590988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.629413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.690940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.631656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.890716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.882309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.131387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.004404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.054195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.326824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.806834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.225639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.299387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.322145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.336291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.343793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.573960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.448437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.602956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.803098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.822339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.748470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.751574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.718846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.788561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.703431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.881245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.922726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.900358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.268328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.140365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.194988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.379859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.432483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.376331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.485513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.750449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.841662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.944260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.197447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.528292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.954815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.816401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.645513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.651076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.154521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.871911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.583478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.053638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.704122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.624368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.607189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.578520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.888812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.942410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.816247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.689783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.636883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.648774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.591869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.545884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.665143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.517987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.414060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.255964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.289468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.132836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.090070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.939771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.800662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.787469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.673642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.683337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.706271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.660887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.603903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.641324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.696139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.544475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.475928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.304505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.381080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.424296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.379150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.404494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.413939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.431372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.469192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.550036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.507824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.477538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.477888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.463606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.339923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.275847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.244057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.276591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.201787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.115873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.039645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.963116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.933739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.859741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.746141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.694485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.676099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.421909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.426424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.372289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.308439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.231663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.163161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.141932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.015532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.162965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.074896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.128599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.949456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.786791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.666584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.581983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.376375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.234657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.076450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.972198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.940114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.872110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.889269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.954680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.864055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.761316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.576620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.474359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.439875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.362761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.201828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.173594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.171348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.072590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.051933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.137978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.983665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.930540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.834455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.695361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.475258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.312427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.109863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.896660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.765664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.550303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.424641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.324473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.268375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.199738] LustreError: 173961:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6274.213960] LustreError: 172070:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6704.608437] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6704.619368] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6704.634436] Lustre: Skipped 475 previous similar messages [ 6704.641991] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6708.707120] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6708.707666] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6708.714958] Lustre: Skipped 1 previous similar message [ 6710.465854] Lustre: server umount lustre-OST0000 complete [ 6716.388205] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6716.402348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6718.953688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6718.959544] Lustre: Skipped 1 previous similar message [ 6720.892090] Lustre: server umount lustre-MDT0000 complete [ 6724.064284] LustreError: 172050:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6724.085556] LustreError: 172050:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6724.230363] LustreError: 172032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788159013 with bad export cookie 16184873308739464410 [ 6724.237266] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6724.252888] LustreError: 172032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6724.541831] Lustre: server umount lustre-MDT0001 complete [ 6734.529616] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 6736.858474] Key type lgssc unregistered [ 6737.139657] LNet: 227260:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6737.146485] LNetError: 227260:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6737.163767] LNet: Removed LNI 192.168.206.125@tcp [ 6737.901917] Key type .llcrypt unregistered [ 6737.904094] Key type ._llcrypt unregistered [ 6759.365703] Key type ._llcrypt registered [ 6759.378076] Key type .llcrypt registered [ 6759.525287] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 6760.605275] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6760.617985] alg: No test for adler32 (adler32-zlib) [ 6761.702375] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 6761.968597] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 6763.689591] Key type lgssc registered [ 6764.868946] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6772.032902] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6772.045076] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6773.569202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6778.381669] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 6782.945349] LustreError: 228684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6782.984775] LustreError: 228684:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6786.628303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6786.968451] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6791.405235] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 6798.460133] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6803.873439] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6811.081540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6811.534113] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6818.005299] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 6818.741476] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6826.367848] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6830.824536] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6832.479815] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6832.499656] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6832.508885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.525088] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6832.525410] LustreError: 228703:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6838.104973] Lustre: lustre-OST0000: Not available for connect from 192.168.206.25@tcp (stopping) [ 6838.249737] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6838.258930] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6839.266350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6839.267318] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6839.276123] Lustre: Skipped 1 previous similar message [ 6840.681992] Lustre: server umount lustre-OST0000 complete [ 6852.580213] LustreError: 228703:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6874.790976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6875.017654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6879.147524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.166905] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6881.375808] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 6888.946876] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6889.446626] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.467352] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6890.505507] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6896.011705] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a2f58860800.ost_server_uuid 50 [ 6897.416597] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9a2f58860800.ost_server_uuid in FULL state after 0 sec [ 6902.752655] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6902.767821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6902.787389] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6903.667396] Lustre: server umount lustre-OST0000 complete [ 6909.921714] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6909.928509] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6909.937097] Lustre: Skipped 1 previous similar message [ 6909.948934] Lustre: Skipped 1 previous similar message [ 6913.502470] Lustre: server umount lustre-MDT0000 complete [ 6915.041183] LustreError: 228683:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6915.059412] LustreError: 228683:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6916.818173] LustreError: 228665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788159206 with bad export cookie 7315634736556142046 [ 6916.831199] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6927.533522] Lustre: server umount lustre-MDT0001 complete [ 6939.990261] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 6943.913177] Key type lgssc unregistered [ 6944.494810] LNet: 233892:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6944.504954] LNetError: 233892:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6944.524895] LNet: Removed LNI 192.168.206.125@tcp [ 6945.684961] Key type .llcrypt unregistered [ 6945.693233] Key type ._llcrypt unregistered [ 6975.126852] Key type ._llcrypt registered [ 6975.131435] Key type .llcrypt registered [ 6975.223718] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 6976.352169] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6976.411641] alg: No test for adler32 (adler32-zlib) [ 6977.416175] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 6977.697943] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 6979.351153] Key type lgssc registered [ 6980.756215] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6991.667289] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6991.696824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6993.128979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6997.477558] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7003.616776] LustreError: 235622:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7003.625500] LustreError: 235622:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7005.706365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7006.160717] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7010.291065] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7013.217801] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7020.764955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7021.080996] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7026.741843] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7034.608972] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7034.785433] Lustre: lustre-OST0001: new disk, initializing [ 7034.788920] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7034.794657] Lustre: 237929:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7034.886420] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7037.971564] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7041.809852] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7043.208333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7043.231193] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7043.465468] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7054.200545] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7059.337767] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7074.278392] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7074.295719] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7074.297017] Lustre: Skipped 2 previous similar messages [ 7074.314441] Lustre: Skipped 3 previous similar messages [ 7079.394655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7079.410604] Lustre: Skipped 3 previous similar messages [ 7084.524526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7084.534094] Lustre: Skipped 3 previous similar messages [ 7087.073943] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7087.434990] Lustre: server umount lustre-MDT0000 complete [ 7089.638065] LustreError: 235618:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7089.663674] LustreError: 235618:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7091.491167] LustreError: 235603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788159380 with bad export cookie 3693093686866481946 [ 7091.505137] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7091.511499] LustreError: 235603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7091.893194] Lustre: server umount lustre-MDT0001 complete [ 7105.981054] Lustre: server umount lustre-OST0000 complete [ 7119.200353] Lustre: server umount lustre-OST0001 complete [ 7139.283383] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 7149.514978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7150.081802] LustreError: 241038:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7150.113840] LustreError: 241038:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7150.165718] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7154.533721] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7155.168395] LustreError: 241039:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7160.294635] LustreError: 241038:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7161.700898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7162.058531] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7165.606694] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7168.450673] Lustre: 242181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7174.263774] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7175.592485] LustreError: 242532:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7175.607404] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7179.857692] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7186.598217] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7186.779164] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7186.785122] Lustre: Skipped 1 previous similar message [ 7187.850365] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7191.789627] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7197.967341] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7201.112586] Lustre: 244050:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7208.420446] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7208.430571] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7208.447627] Lustre: Skipped 2 previous similar messages [ 7208.464053] Lustre: Skipped 3 previous similar messages [ 7210.048963] Lustre: server umount mylustre-MDT0000 complete [ 7212.726816] LustreError: 241013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788159501 with bad export cookie 3693093686866484753 [ 7212.731556] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7212.736890] LustreError: 241013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7212.993211] Lustre: server umount mylustre-MDT0001 complete [ 7225.128309] Lustre: server umount mylustre-OST0000 complete [ 7238.866843] Lustre: server umount mylustre-OST0001 complete [ 7262.302103] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 7271.306287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7271.790443] LustreError: 246356:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7271.812477] LustreError: 246356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7271.895408] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7276.641476] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7284.785186] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7289.841294] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7292.798510] Lustre: 247497:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7299.757735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7306.281154] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7310.380365] LustreError: 247850:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7310.424718] LustreError: 247850:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7314.600673] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7315.862848] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7316.929747] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7320.675236] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7328.571054] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7331.803453] Lustre: 249366:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7341.537424] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7341.549778] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7341.563768] Lustre: Skipped 2 previous similar messages [ 7341.580947] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7344.736666] Lustre: server umount tfs-MDT0000 complete [ 7349.362835] LustreError: 246333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788159638 with bad export cookie 3693093686866486643 [ 7349.381732] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7349.395495] LustreError: 246333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7349.784459] Lustre: server umount tfs-MDT0001 complete [ 7364.422491] Lustre: server umount tfs-OST0000 complete [ 7378.701710] Lustre: server umount tfs-OST0001 complete [ 7405.130523] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 7418.297093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7418.720727] LustreError: 251632:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7418.757529] LustreError: 251632:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 7418.844226] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7418.848932] Lustre: Skipped 3 previous similar messages [ 7423.699392] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7433.905044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7439.042316] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7442.215290] Lustre: 252772:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7448.179207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7454.101243] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7462.323577] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7463.536637] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7463.545684] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7467.720036] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7474.670209] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7477.660529] Lustre: 254644:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7482.957502] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:02:51 (1788159771) [ 7499.749502] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7499.752206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7499.774194] Lustre: Skipped 5 previous similar messages [ 7499.792673] Lustre: Skipped 7 previous similar messages [ 7504.249205] Lustre: server umount lustre-MDT0000 complete [ 7507.702392] LustreError: 251607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788159796 with bad export cookie 3693093686866488309 [ 7507.706990] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7507.721401] LustreError: 251607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7509.996213] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7510.012079] Lustre: Skipped 2 previous similar messages [ 7516.067362] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7516.072472] Lustre: Skipped 4 previous similar messages [ 7522.271820] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7522.598433] Lustre: server umount lustre-MDT0001 complete [ 7540.191958] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7540.289301] Lustre: server umount lustre-OST0000 complete [ 7549.836068] Lustre: server umount lustre-OST0001 complete [ 7555.134502] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 7564.253628] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 7596.250932] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7596.250932] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7596.250932] [ 7596.302956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7596.512136] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7596.556731] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7596.625288] Lustre: lustre-MDT0000: new disk, initializing [ 7596.670411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7596.674406] Lustre: Skipped 3 previous similar messages [ 7596.682672] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7600.829367] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7610.502812] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7610.502812] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7610.502812] [ 7610.547602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7610.669098] Lustre: 257830:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7610.725626] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7610.729641] Lustre: Skipped 1 previous similar message [ 7610.797857] Lustre: lustre-MDT0001: new disk, initializing [ 7610.951957] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7610.966841] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7614.711681] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7623.572081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7623.833269] Lustre: lustre-OST0000: new disk, initializing [ 7623.836520] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7623.846339] Lustre: 259414:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7625.112528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7625.120529] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7625.259064] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7629.721882] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7639.551139] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7639.705619] Lustre: lustre-OST0001: new disk, initializing [ 7639.714813] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7639.725687] Lustre: 260282:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7641.676375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7641.683816] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7641.727533] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7644.927632] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7654.110982] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:05:42 (1788159942) [ 7666.305282] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:05:54 (1788159954) [ 7681.506827] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7681.518980] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7681.521788] LustreError: Skipped 1 previous similar message [ 7681.541524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7681.556847] Lustre: Skipped 5 previous similar messages [ 7695.327918] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7695.493043] Lustre: server umount lustre-OST0000 complete [ 7698.402143] LustreError: 259397:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7698.426523] LustreError: 259397:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 19 previous similar messages [ 7703.010211] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7703.023629] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7703.041234] Lustre: Skipped 1 previous similar message [ 7705.069602] Lustre: server umount lustre-MDT0000 complete [ 7708.124153] LustreError: 257823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788159997 with bad export cookie 3693093686866489856 [ 7708.125957] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7708.136971] LustreError: 257823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7714.525509] Lustre: server umount lustre-MDT0001 complete [ 7726.983688] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 7743.927532] Lustre: server umount lustre-OST0001 complete [ 7751.041847] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 7768.090691] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 7808.568282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7808.767424] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7808.824372] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7808.932211] Lustre: lustre-MDT0000: new disk, initializing [ 7809.108591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7813.567915] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7823.283168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7823.405431] Lustre: 264844:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7823.695272] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7828.617913] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7837.962695] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7843.404096] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7850.066848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7850.461822] Lustre: lustre-OST0000: new disk, initializing [ 7850.469750] Lustre: Skipped 1 previous similar message [ 7850.489260] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7850.500377] Lustre: Skipped 2 previous similar messages [ 7850.511463] Lustre: 266813:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7852.346176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7852.358145] Lustre: Skipped 1 previous similar message [ 7852.364395] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7852.497403] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7857.076606] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7867.558818] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7878.112292] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7878.120199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7878.123621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7878.128956] Lustre: Skipped 4 previous similar messages [ 7878.134128] Lustre: Skipped 11 previous similar messages [ 7881.613979] Lustre: server umount lustre-OST0000 complete [ 7892.321823] Lustre: server umount lustre-MDT0000 complete [ 7896.982510] LustreError: 264836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788160186 with bad export cookie 3693093686866491333 [ 7896.986304] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7897.001133] LustreError: 264836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7897.591816] Lustre: server umount lustre-MDT0001 complete [ 7908.899939] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 7912.212498] Key type lgssc unregistered [ 7912.501834] LNet: 268572:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7912.512566] LNetError: 268572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7912.540838] LNet: Removed LNI 192.168.206.125@tcp [ 7913.751410] Key type .llcrypt unregistered [ 7913.759988] Key type ._llcrypt unregistered [ 7939.271561] Key type ._llcrypt registered [ 7939.275564] Key type .llcrypt registered [ 7939.418823] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 7940.774479] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7940.867953] alg: No test for adler32 (adler32-zlib) [ 7942.056842] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 7942.299423] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 7944.047224] Key type lgssc registered [ 7945.806753] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7954.172784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7954.195663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7955.571828] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7959.872748] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7965.168959] LustreError: 270034:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7965.194513] LustreError: 270034:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7967.652924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7968.001034] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7972.077420] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 7978.690404] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7984.469860] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7990.992689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7991.223485] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7997.421654] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8000.430310] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8003.889943] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8004.941987] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8008.672402] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8008.676404] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8008.700515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8010.720252] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8010.722338] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8010.744180] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8013.237378] Lustre: server umount lustre-OST0000 complete [ 8019.424595] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8019.437431] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8019.463266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8019.483734] Lustre: Skipped 1 previous similar message [ 8023.544314] Lustre: server umount lustre-MDT0000 complete [ 8026.080436] LustreError: 270035:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8026.131694] LustreError: 270035:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8027.725191] LustreError: 270016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788160316 with bad export cookie 529965431365577311 [ 8027.737162] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8027.745633] LustreError: 270016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8028.186652] Lustre: server umount lustre-MDT0001 complete [ 8040.749626] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 8044.102545] Key type lgssc unregistered [ 8044.398462] LNet: 273621:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8044.406450] LNetError: 273621:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8044.415294] LNet: Removed LNI 192.168.206.125@tcp [ 8045.279941] Key type .llcrypt unregistered [ 8045.282823] Key type ._llcrypt unregistered [ 8059.268781] Key type ._llcrypt registered [ 8059.271341] Key type .llcrypt registered [ 8059.349202] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 8060.288023] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8060.322276] alg: No test for adler32 (adler32-zlib) [ 8061.315534] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 8061.528184] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 8063.175150] Key type lgssc registered [ 8063.952438] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8071.072272] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8071.085910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8086.560354] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8091.679282] LustreError: 274809:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.125@tcp: failed processing log, type 4: rc = -110 [ 8118.367201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8123.885812] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8131.004201] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8133.376903] Lustre: server umount lustre-OST0000 complete [ 8151.387440] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 8153.775911] Key type lgssc unregistered [ 8154.024080] LNet: 276440:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8154.032759] LNetError: 276440:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8154.070824] LNet: Removed LNI 192.168.206.125@tcp [ 8154.747969] Key type .llcrypt unregistered [ 8154.752461] Key type ._llcrypt unregistered [ 8190.194659] Key type ._llcrypt registered [ 8190.199694] Key type .llcrypt registered [ 8190.300121] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 8191.263617] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8191.543669] alg: No test for adler32 (adler32-zlib) [ 8192.663815] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 8192.882532] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 8194.535151] Key type lgssc registered [ 8195.604594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8207.445454] loop0: detected capacity change from 0 to 204800000 [ 8207.642629] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8207.660955] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8208.068738] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8209.103882] loop0: detected capacity change from 0 to 204800000 [ 8209.211817] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8209.227389] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8209.516355] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8210.462852] loop0: detected capacity change from 0 to 204800000 [ 8210.662229] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8210.677867] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8210.765918] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8211.713532] loop0: detected capacity change from 0 to 204800000 [ 8211.899700] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8213.069321] loop0: detected capacity change from 0 to 204800000 [ 8213.105855] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8217.080669] loop0: detected capacity change from 0 to 204800000 [ 8217.105317] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8221.198306] loop0: detected capacity change from 0 to 204800000 [ 8221.246386] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8224.951552] loop0: detected capacity change from 0 to 204800000 [ 8224.977581] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8228.759339] loop0: detected capacity change from 0 to 204800000 [ 8228.855971] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8228.932762] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8228.992670] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8228.999802] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8229.007195] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8229.017167] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8229.028256] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8229.038681] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8229.047073] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8229.055323] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8229.062881] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8229.070243] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8229.078302] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8229.086955] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8229.096716] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8229.108551] Lustre: 279858:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8234.952342] Lustre: server umount MGS complete [ 8235.892209] loop0: detected capacity change from 0 to 204800000 [ 8235.988241] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8236.497626] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8236.546535] Lustre: lustre-MDT0000: Aborting client recovery [ 8237.230826] loop1: detected capacity change from 0 to 204800000 [ 8237.368542] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8237.433556] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8237.443531] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8237.462041] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8237.473149] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8237.483924] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8237.492711] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8237.503940] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8237.519966] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8237.640595] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8237.679599] Lustre: lustre-MDT0001: Aborting client recovery [ 8238.510896] loop2: detected capacity change from 0 to 204800000 [ 8238.602578] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8238.654092] Lustre: 280369:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8238.662199] Lustre: 280369:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8238.674593] Lustre: 280369:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8238.687506] Lustre: 280369:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8238.696511] Lustre: 280369:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8238.708644] Lustre: 280369:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8238.719951] Lustre: 280369:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8238.801812] Lustre: Found index 0 for lustre-OST0000, updating log [ 8238.827410] Lustre: 280388:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8238.886629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8238.901279] Lustre: lustre-OST0000: Aborting recovery [ 8239.823332] loop3: detected capacity change from 0 to 204800000 [ 8240.028422] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8240.086681] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8240.099364] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8240.107574] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8240.116671] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8240.130682] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8240.140406] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8240.150684] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8240.172789] Lustre: Found index 1 for lustre-OST0001, updating log [ 8240.208959] Lustre: 280458:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8240.313910] Lustre: lustre-OST0001: Aborting recovery [ 8245.788147] LustreError: 280235:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8246.761305] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8246.769421] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8246.834316] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8246.859939] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8247.782802] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8247.782970] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8248.194364] LustreError: 280202:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3362us: rc = -2 [ 8248.722313] LustreError: 280203:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 399us: rc = -2 [ 8248.755455] LustreError: 280203:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8263.138069] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8263.138988] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8263.151226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8263.180739] Lustre: Skipped 3 previous similar messages [ 8267.653309] Lustre: server umount lustre-MDT0000 complete [ 8268.257391] LustreError: 280204:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8268.271836] LustreError: 280204:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 8268.543617] LustreError: 280184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788160557 with bad export cookie 16949425359360132180 [ 8268.569798] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8273.385669] LustreError: 280200:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8273.387612] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8273.391709] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8273.391722] Lustre: Skipped 3 previous similar messages [ 8273.408203] LustreError: 280200:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8273.456959] Lustre: Skipped 1 previous similar message [ 8275.037961] Lustre: server umount lustre-MDT0001 complete [ 8282.448461] Lustre: server umount lustre-OST0000 complete [ 8289.659879] Lustre: server umount lustre-OST0001 complete [ 8298.688992] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:16:26 (1788160586) [ 8323.026344] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 8332.390871] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 8368.637260] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8368.978684] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8369.043474] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8369.188467] Lustre: lustre-MDT0000: new disk, initializing [ 8369.288815] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8369.296142] Lustre: Skipped 1 previous similar message [ 8369.325599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8374.351565] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8385.895897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8385.988164] Lustre: 284528:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8386.035089] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8386.038318] Lustre: Skipped 1 previous similar message [ 8386.101232] Lustre: lustre-MDT0001: new disk, initializing [ 8386.161747] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8386.189153] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8386.197337] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8390.387069] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8398.872886] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8404.656817] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8411.493222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8411.711732] Lustre: lustre-OST0000: new disk, initializing [ 8411.715398] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8411.720716] Lustre: 286495:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8411.800423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8417.312039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8417.318544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8417.452679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8417.592103] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8425.908344] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8430.650839] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8442.226551] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8449.169483] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8453.315380] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8494.048908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8494.062564] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8494.079232] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8494.086316] Lustre: Skipped 1 previous similar message [ 8499.682722] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8499.701182] Lustre: Skipped 2 previous similar messages [ 8504.823925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8504.837978] Lustre: Skipped 1 previous similar message [ 8507.871113] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8508.046738] Lustre: server umount lustre-OST0000 complete [ 8514.016871] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8514.025478] LustreError: Skipped 1 previous similar message [ 8514.035047] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8514.051635] Lustre: Skipped 1 previous similar message [ 8514.056843] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8517.411327] Lustre: server umount lustre-MDT0000 complete [ 8520.162082] LustreError: 284535:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8520.181765] LustreError: 284535:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8520.232386] LustreError: 284520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788160809 with bad export cookie 16949425359360134882 [ 8520.239400] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8520.241403] LustreError: 284520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8520.557802] Lustre: server umount lustre-MDT0001 complete [ 8528.271450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8532.023200] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8536.796098] Lustre: server umount MGS complete [ 8547.569961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8548.177049] LustreError: 290457:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8548.190732] LustreError: 290470:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8548.313936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8553.457039] LustreError: 290458:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8553.725879] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8563.050647] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8568.972564] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8577.280240] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8582.962306] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8589.925796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8590.252094] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8590.260147] Lustre: Skipped 1 previous similar message [ 8595.562622] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8602.051716] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8634.347060] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8634.347154] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8634.358179] LustreError: Skipped 1 previous similar message [ 8634.358861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8634.371288] Lustre: Skipped 2 previous similar messages [ 8634.388274] Lustre: Skipped 3 previous similar messages [ 8647.135120] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8647.419690] Lustre: server umount lustre-OST0000 complete [ 8654.818284] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8658.180582] Lustre: server umount lustre-MDT0000 complete [ 8659.953131] LustreError: 290453:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8659.978478] LustreError: 290453:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8661.804204] LustreError: 290438:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788160950 with bad export cookie 16949425359360136709 [ 8661.814679] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8661.817333] LustreError: 290438:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8662.359227] Lustre: server umount lustre-MDT0001 complete [ 8673.204770] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 8676.240227] Key type lgssc unregistered [ 8676.486602] LNet: 294406:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8676.501727] LNetError: 294406:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8676.529620] LNet: Removed LNI 192.168.206.125@tcp [ 8677.564212] Key type .llcrypt unregistered [ 8677.567632] Key type ._llcrypt unregistered [ 8711.760473] Key type ._llcrypt registered [ 8711.768123] Key type .llcrypt registered [ 8711.851958] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 8725.632188] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 8726.699841] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8726.769068] alg: No test for adler32 (adler32-zlib) [ 8727.808151] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 8728.184894] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 8729.919172] Key type lgssc registered [ 8730.906327] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8764.402377] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8764.425218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8765.634617] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8765.652522] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8765.712701] Lustre: lustre-MDT0000: new disk, initializing [ 8765.800640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8765.816397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8769.660257] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8779.529853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8779.623570] Lustre: 298353:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8779.694678] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8779.699265] Lustre: Skipped 1 previous similar message [ 8779.775991] Lustre: lustre-MDT0001: new disk, initializing [ 8779.871518] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8779.914642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8779.932893] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8784.106331] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8794.174378] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8800.245342] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8806.701636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8807.031604] Lustre: lustre-OST0000: new disk, initializing [ 8807.034991] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8807.042479] Lustre: 300323:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8807.163570] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8813.649382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8813.684109] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8813.867277] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8814.754340] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8824.257776] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8830.242971] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8840.536075] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8849.534886] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8895.975453] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8895.979376] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8895.982788] LustreError: Skipped 1 previous similar message [ 8895.984294] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8896.014350] Lustre: Skipped 1 previous similar message [ 8901.091610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8901.098170] Lustre: Skipped 2 previous similar messages [ 8906.211112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8906.222345] Lustre: Skipped 1 previous similar message [ 8909.279153] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8909.504703] Lustre: server umount lustre-OST0000 complete [ 8916.450706] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8916.454153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8916.468581] Lustre: Skipped 1 previous similar message [ 8916.480100] Lustre: Skipped 1 previous similar message [ 8919.543732] Lustre: server umount lustre-MDT0000 complete [ 8923.418802] LustreError: 298346:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788161212 with bad export cookie 11951281921303953113 [ 8923.423650] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8923.429956] LustreError: 298346:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8923.819760] Lustre: server umount lustre-MDT0001 complete [ 8931.635569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8935.488811] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8939.702657] Lustre: server umount MGS complete [ 8949.151670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8949.458765] LustreError: 304289:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8949.476922] LustreError: 304289:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8949.491847] LustreError: 304302:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8949.542717] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8953.329694] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8954.854762] LustreError: 304290:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8958.945987] LustreError: 304289:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8960.988604] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8961.374318] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8965.061965] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8971.269461] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8976.150141] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8982.629884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8982.969047] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8988.605698] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 8995.417691] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9026.024768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9026.036143] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9026.040798] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9026.056686] Lustre: Skipped 1 previous similar message [ 9029.602147] Lustre: server umount lustre-OST0000 complete [ 9033.186788] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9033.193139] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9033.201095] Lustre: Skipped 1 previous similar message [ 9036.261122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9036.270387] Lustre: Skipped 2 previous similar messages [ 9039.076743] Lustre: server umount lustre-MDT0000 complete [ 9041.377142] LustreError: 304290:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9041.401611] LustreError: 304290:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9042.418674] LustreError: 304271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788161331 with bad export cookie 11951281921303954947 [ 9042.433072] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9042.438546] LustreError: 304271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9042.816627] Lustre: server umount lustre-MDT0001 complete [ 9054.979284] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [ 9057.799647] Key type lgssc unregistered [ 9058.204928] LNet: 308232:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9058.218934] LNetError: 308232:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9059.247900] LNet: Removed LNI 192.168.206.125@tcp [ 9060.400568] Key type .llcrypt unregistered [ 9060.402483] Key type ._llcrypt unregistered [ 9083.613276] Key type ._llcrypt registered [ 9083.614933] Key type .llcrypt registered [ 9083.700381] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [ 9099.723197] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 9101.532512] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9101.569885] alg: No test for adler32 (adler32-zlib) [ 9102.722240] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [ 9103.093263] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [ 9104.903152] Key type lgssc registered [ 9106.344136] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9134.890313] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [ 9147.961409] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9147.983212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9149.223685] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9149.264223] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9149.355342] Lustre: lustre-MDT0000: new disk, initializing [ 9149.463377] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9149.492962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9153.471449] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [ 9162.790353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9163.013524] Lustre: lustre-OST0000: new disk, initializing [ 9163.017543] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9163.021244] Lustre: Skipped 1 previous similar message [ 9163.025210] Lustre: 312283:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9163.084443] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9167.951128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9167.957176] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9167.987654] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9168.305306] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10002.664269] Lustre: server umount lustre-MDT0000 complete [10008.072251] LustreError: 311470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788162297 with bad export cookie 5091966524114689110 [10008.100763] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10018.527549] Lustre: server umount lustre-OST0000 complete [10048.196448] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [10058.625310] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [10093.051971] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10093.255843] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10093.295655] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10093.355438] Lustre: lustre-MDT0000: new disk, initializing [10093.430913] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10093.446389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10097.702669] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10107.303438] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10107.450688] Lustre: 316618:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10107.514652] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10107.519341] Lustre: Skipped 1 previous similar message [10107.583220] Lustre: lustre-MDT0001: new disk, initializing [10107.685071] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10107.730923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10107.747428] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10112.745789] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10121.827991] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10127.683390] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10134.420913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10134.653240] Lustre: lustre-OST0000: new disk, initializing [10134.656774] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10134.664243] Lustre: 318590:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10134.777892] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10140.567961] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10144.821783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10144.833246] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10144.917795] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10148.890909] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10152.620746] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10152.830782] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10156.808414] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10157.025504] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10160.608282] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10160.611423] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10160.623757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10160.628674] Lustre: Skipped 1 previous similar message [10160.643653] Lustre: Skipped 1 previous similar message [10164.713913] Lustre: server umount lustre-OST0000 complete [10169.312396] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10169.320748] LustreError: Skipped 1 previous similar message [10169.324500] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10169.340213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10170.851904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10170.853188] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10173.835437] Lustre: server umount lustre-MDT0000 complete [10176.762214] LustreError: 317414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788162465 with bad export cookie 5091966524117629929 [10176.769646] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10176.777980] LustreError: 317414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10177.046252] Lustre: server umount lustre-MDT0001 complete [10180.087906] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10181.569585] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:47:49 (1788162469) [10187.398939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10187.728563] LustreError: 320901:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10187.743755] LustreError: 320901:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10187.786460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10191.447596] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10192.872160] LustreError: 320902:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10196.963903] LustreError: 320901:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10198.099855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10198.415232] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10202.084538] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10208.610299] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10214.576881] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10221.095663] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10221.581651] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10229.034427] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10237.006301] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10244.716727] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10244.964517] Lustre: lustre-OST0001: new disk, initializing [10244.968729] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10244.981993] Lustre: 323736:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10245.088251] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10251.887932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10251.901268] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10251.985713] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10252.011096] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10252.287748] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10253.031257] LustreError: 324069:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10254.120481] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10254.141219] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10256.168550] LustreError: 324069:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10256.180865] LustreError: 324069:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10260.320856] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10260.349664] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10262.769693] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10263.073952] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10268.318981] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a2f483c2000.ost_server_uuid 50 [10268.592108] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10268.611188] LustreError: 324062:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [10269.736851] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9a2f483c2000.ost_server_uuid in FULL state after 0 sec [10275.191905] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9a2f483c2000.ost_server_uuid 50 [10276.568947] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9a2f483c2000.ost_server_uuid in FULL state after 0 sec [10284.010160] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10293.225626] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10293.230545] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10293.241126] Lustre: Skipped 1 previous similar message [10293.259833] Lustre: Skipped 2 previous similar messages [10297.570680] Lustre: server umount lustre-OST0001 complete [10298.343925] LustreError: 322723:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10298.359891] LustreError: 322723:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10303.463933] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10303.473786] LustreError: 322724:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10303.475883] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10303.475890] Lustre: Skipped 1 previous similar message [10303.485933] Lustre: Skipped 1 previous similar message [10303.505035] LustreError: 322724:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10309.588200] Lustre: server umount lustre-OST0000 complete [10313.697511] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10313.702995] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10313.713610] Lustre: Skipped 1 previous similar message [10313.732518] Lustre: Skipped 4 previous similar messages [10319.888208] Lustre: server umount lustre-MDT0000 complete [10323.937893] LustreError: 320901:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10323.968848] LustreError: 320901:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10324.032810] LustreError: 321635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788162613 with bad export cookie 5091966524117630811 [10324.048570] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10324.051774] LustreError: 321635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10324.435552] Lustre: server umount lustre-MDT0001 complete [10338.030671] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [10341.379238] Key type lgssc unregistered [10341.664623] LNet: 326424:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10341.673839] LNetError: 326424:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10341.720404] LNet: Removed LNI 192.168.206.125@tcp [10342.606325] Key type .llcrypt unregistered [10342.608168] Key type ._llcrypt unregistered [10365.544599] Key type ._llcrypt registered [10365.547782] Key type .llcrypt registered [10365.677990] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [10366.505108] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10366.733366] alg: No test for adler32 (adler32-zlib) [10367.912236] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [10368.211842] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [10369.887269] Key type lgssc registered [10371.158290] Lustre: Echo OBD driver; http://www.lustre.org/ [10378.411160] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10378.425514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10379.893531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10384.094860] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10390.499622] LustreError: 327851:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10390.518779] LustreError: 327851:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10392.275877] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10395.624797] LustreError: 327850:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10400.160605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10400.795123] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10406.417481] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10416.725629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10417.240284] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10424.644472] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10425.465691] LustreError: 329531:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10425.503694] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10430.961634] LustreError: 329531:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10430.999234] LustreError: 329531:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10433.241874] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10436.085358] LustreError: 329927:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10436.122941] LustreError: 329927:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10442.302435] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10442.873361] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10448.403529] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10452.411459] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10462.416179] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10469.019311] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10469.282690] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10571.240061] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10571.252110] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10571.254215] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10571.264811] LustreError: Skipped 1 previous similar message [10571.284865] Lustre: Skipped 1 previous similar message [10575.867046] Lustre: server umount lustre-OST0001 complete [10576.356141] LustreError: 329529:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10576.392689] LustreError: 329529:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10584.545189] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10584.549391] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10584.553972] LustreError: Skipped 1 previous similar message [10584.559070] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10584.582162] Lustre: Skipped 2 previous similar messages [10586.604721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10586.612894] Lustre: Skipped 2 previous similar messages [10588.867515] Lustre: server umount lustre-OST0000 complete [10595.298696] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10595.308441] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10595.331441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10599.026339] Lustre: server umount lustre-MDT0000 complete [10601.963311] LustreError: 328964:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10601.998373] LustreError: 328964:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [10602.808672] LustreError: 327829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788162891 with bad export cookie 9436375122155696484 [10602.812260] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10602.818799] LustreError: 327829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10603.411460] Lustre: server umount lustre-MDT0001 complete [10615.149896] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [10618.052748] Key type lgssc unregistered [10618.312947] LNet: 332793:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10618.320381] LNetError: 332793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10618.348374] LNet: Removed LNI 192.168.206.125@tcp [10619.430850] Key type .llcrypt unregistered [10619.434778] Key type ._llcrypt unregistered [10665.686866] Key type ._llcrypt registered [10665.688537] Key type .llcrypt registered [10665.835691] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [10667.149174] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10667.334835] alg: No test for adler32 (adler32-zlib) [10668.492216] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [10668.806776] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [10670.535184] Key type lgssc registered [10671.806642] Lustre: Echo OBD driver; http://www.lustre.org/ [10681.815711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10681.862948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10683.549245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10688.201411] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10693.100092] LustreError: 335085:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10693.117839] LustreError: 335085:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10697.150553] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10697.506986] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10702.110643] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10705.368981] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10713.359602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10713.671083] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10720.067763] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10721.830679] LustreError: 336573:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10726.896227] LustreError: 336574:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10727.935153] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10728.999152] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10729.237214] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10734.591444] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10736.481295] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10745.813876] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10757.285472] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10774.496050] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10774.504659] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10774.520421] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10775.522247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10775.525200] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10775.544062] Lustre: Skipped 3 previous similar messages [10777.313352] Lustre: server umount lustre-MDT0000 complete [10780.648513] LustreError: 335080:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10780.680721] LustreError: 335080:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [10781.435072] LustreError: 335065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788163070 with bad export cookie 16629475442926815654 [10781.440276] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10781.447894] LustreError: 335065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10781.892163] Lustre: server umount lustre-MDT0001 complete [10796.252359] Lustre: server umount lustre-OST0000 complete [10811.153118] Lustre: server umount lustre-OST0001 complete [10830.120711] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [10839.147841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10839.565877] LustreError: 340348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10839.637027] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10843.744635] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10849.769978] LustreError: 340349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10849.793654] LustreError: 340349:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10851.524632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10851.783516] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10855.793662] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10858.412217] Lustre: 341486:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10865.298225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10871.401277] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10876.077453] LustreError: 341840:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10879.943719] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10880.237240] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10880.248311] Lustre: Skipped 1 previous similar message [10884.350769] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10884.352779] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10886.507209] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [10893.677894] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10897.027753] Lustre: 343361:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10903.008967] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10903.019042] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10903.035953] Lustre: Skipped 2 previous similar messages [10903.041820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10905.063511] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10905.064257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10905.082123] Lustre: Skipped 1 previous similar message [10905.100812] Lustre: Skipped 3 previous similar messages [10906.948874] Lustre: server umount lustre-MDT0000 complete [10910.180210] LustreError: 342445:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10910.201603] LustreError: 342445:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10910.308263] LustreError: 341489:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788163199 with bad export cookie 16629475442926817355 [10910.317490] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10910.325707] LustreError: 341489:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10910.747218] Lustre: server umount lustre-MDT0001 complete [10924.624052] Lustre: server umount lustre-OST0000 complete [10937.212904] Lustre: server umount lustre-OST0001 complete [10941.021744] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10942.557633] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10943.977854] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:00:32 (1788163232) [10964.642314] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [10973.553724] loop0: detected capacity change from 0 to 1099511627776 [10974.395636] SGI XFS with ACLs, security attributes, quota, no debug enabled [10974.420642] *** VALIDATE xfs *** [10974.445237] XFS (loop0): Mounting V5 Filesystem [10974.485883] XFS (loop0): Ending clean mount [10975.788955] loop1: detected capacity change from 0 to 18691697672192 [11084.169579] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [11093.204964] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [11125.426328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11125.622470] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11125.654681] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11125.771920] Lustre: lustre-MDT0000: new disk, initializing [11125.866833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11125.887835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11129.978900] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11139.389076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11139.529785] Lustre: 348506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11139.577348] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11139.589824] Lustre: Skipped 1 previous similar message [11139.667843] Lustre: lustre-MDT0001: new disk, initializing [11139.788750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11139.808587] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11143.823765] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11152.617306] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11157.641812] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11163.848293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11164.180411] Lustre: lustre-OST0000: new disk, initializing [11164.183312] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11164.193987] Lustre: 350472:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11165.467169] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11165.497592] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11165.655505] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11170.701977] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11182.242358] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11189.085197] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11189.348463] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11194.073409] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11194.277455] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11201.513669] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11201.517897] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11201.548464] Lustre: Skipped 1 previous similar message [11201.562909] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11201.576529] Lustre: Skipped 1 previous similar message [11202.846523] Lustre: server umount lustre-OST0000 complete [11206.624105] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11206.630829] LustreError: Skipped 1 previous similar message [11211.747041] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11211.765589] Lustre: Skipped 3 previous similar messages [11212.654284] Lustre: server umount lustre-MDT0000 complete [11216.311977] LustreError: 348498:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788163505 with bad export cookie 16629475442926818909 [11216.322807] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11216.828254] Lustre: server umount lustre-MDT0001 complete [11225.094560] XFS (loop0): Unmounting Filesystem [11230.126193] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:05:18 (1788163518) [11236.702783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11237.227659] LustreError: 353034:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11237.355221] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11237.367028] Lustre: Skipped 2 previous similar messages [11242.067560] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11249.881971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11254.117945] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11261.315251] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11266.475832] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11273.269118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11279.421181] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11286.264277] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11296.227635] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11296.238958] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11296.258872] Lustre: Skipped 2 previous similar messages [11296.281711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11299.770896] Lustre: server umount lustre-OST0000 complete [11306.464902] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11306.478203] LustreError: Skipped 1 previous similar message [11310.428129] Lustre: server umount lustre-MDT0000 complete [11313.834886] LustreError: 353014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788163603 with bad export cookie 16629475442926819791 [11313.839396] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11313.847030] LustreError: 353014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [11314.227146] Lustre: server umount lustre-MDT0001 complete [11324.863113] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [11327.446261] Key type lgssc unregistered [11327.749435] LNet: 356662:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11327.759311] LNetError: 356662:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11327.778520] LNet: Removed LNI 192.168.206.125@tcp [11328.740158] Key type .llcrypt unregistered [11328.743215] Key type ._llcrypt unregistered [11354.604427] Key type ._llcrypt registered [11354.607820] Key type .llcrypt registered [11354.767560] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [11356.098537] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11356.121850] alg: No test for adler32 (adler32-zlib) [11357.115440] Lustre: Lustre: Build Version: 2.17.55_27_gdd19d92 [11357.380310] LNet: Added LNI 192.168.206.125@tcp [8/256/0/180] [11359.065346] Key type lgssc registered [11360.187474] Lustre: Echo OBD driver; http://www.lustre.org/ [11368.312584] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11368.335656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11369.801258] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11374.221174] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11379.173952] LustreError: 358227:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11379.200074] LustreError: 358227:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11381.559615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11381.950601] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11385.873591] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11392.405892] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11397.445452] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11403.427712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11403.729227] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11409.613533] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11415.820682] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11423.200917] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11423.206102] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11423.221612] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11424.224789] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11424.231357] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11424.240550] Lustre: Skipped 1 previous similar message [11428.658694] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.25@tcp (stopping) [11428.676292] Lustre: Skipped 1 previous similar message [11429.126400] Lustre: server umount lustre-MDT0001 complete [11429.348137] LustreError: 359286:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11429.360951] LustreError: 359286:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11433.776864] LustreError: 359594:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11438.425893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11438.545560] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11438.550821] Lustre: Found index 1 for lustre-MDT0001, updating log [11438.644172] Lustre: 358217:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11439.032600] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11443.411359] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11444.211921] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11447.804862] LustreError: 361826:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11447.812725] LustreError: 361826:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11447.818738] LustreError: 358219:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11447.916523] LustreError: 361829:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.125@tcp: cfg command failed: rc = -17 [11447.922693] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11447.922693] [11751.254896] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.25@tcp (stopping) [11751.394231] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11751.418810] Lustre: Skipped 1 previous similar message [11753.826313] Lustre: server umount lustre-MDT0001 complete [11756.343345] LustreError: 358222:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11756.353204] LustreError: 358222:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11761.457675] LustreError: 359594:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11761.475231] LustreError: 359594:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11761.556775] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11761.652987] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11761.660157] Lustre: Found index 1 for lustre-MDT0001, updating log [11761.698510] Lustre: 358217:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11761.954819] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11765.679537] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [11767.290062] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11770.888387] LustreError: 370962:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11770.901683] LustreError: 370962:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11770.914361] LustreError: 358219:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11770.922926] LustreError: 358219:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11771.001365] LustreError: 370971:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11771.005287] LustreError: 370971:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11771.009335] LustreError: 370971:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.125@tcp: cfg command failed: rc = -17 [11771.026929] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.125@tcp [11771.026929] [12073.812654] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.25@tcp (stopping) [12073.821742] Lustre: Skipped 2 previous similar messages [12074.467568] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12074.470324] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12074.491874] Lustre: Skipped 1 previous similar message [12078.385575] Lustre: server umount lustre-MDT0001 complete [12078.896293] LustreError: 359594:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12078.920931] LustreError: 359594:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12087.866479] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12087.971889] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12087.976540] Lustre: Found index 1 for lustre-MDT0001, updating log [12088.006958] Lustre: 358216:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12088.281953] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12092.467840] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [12093.429390] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12094.958291] LustreError: 380572:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12094.963450] LustreError: 380572:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12094.977794] LustreError: 358219:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12094.991746] LustreError: 358219:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12095.093993] LustreError: 380575:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12095.110836] LustreError: 380575:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.125@tcp: cfg command failed: rc = -114 [12095.127158] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12095.127158] [12415.975599] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12415.990694] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12415.990905] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12415.990988] Lustre: Skipped 1 previous similar message [12416.023729] Lustre: Skipped 4 previous similar messages [12418.492615] Lustre: server umount lustre-MDT0000 complete [12421.106788] LustreError: 358224:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12421.127055] LustreError: 358224:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [12421.537571] LustreError: 358207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788164710 with bad export cookie 3578915963895279429 [12421.554039] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12426.210035] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12426.221745] Lustre: Skipped 1 previous similar message [12427.904654] Lustre: server umount lustre-MDT0001 complete [12431.310981] Lustre: server umount lustre-OST0000 complete [12440.334850] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [12449.133984] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [12481.144493] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:26:09 (1788164769) [12488.742751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12489.020988] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12489.096654] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12489.215842] Lustre: lustre-MDT0000: new disk, initializing [12489.371225] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12489.394105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12493.802104] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [12504.260301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12504.399832] Lustre: 392003:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12504.453984] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12504.464484] Lustre: Skipped 1 previous similar message [12504.540364] Lustre: lustre-MDT0001: new disk, initializing [12504.617241] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12504.661553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12504.673314] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12509.482211] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [12518.412544] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12523.661238] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12530.105978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12530.315841] Lustre: lustre-OST0000: new disk, initializing [12530.318987] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12530.323707] Lustre: 393975:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12530.420368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12536.373383] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_default_debug -1 all [12538.429756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12538.447231] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12538.587315] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12545.682836] Lustre: DEBUG MARKER: oleg625-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12554.218129] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12554.223079] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12554.246300] Lustre: Skipped 1 previous similar message [12554.250969] Lustre: Skipped 1 previous similar message [12555.263433] Lustre: server umount lustre-MDT0000 complete [12559.169221] LustreError: 392810:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12559.171103] LustreError: 393974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788164848 with bad export cookie 3578915963895281382 [12559.175340] LustreError: MGC192.168.206.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12559.233086] LustreError: 393974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [12559.331349] LustreError: 393614:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12559.345628] LustreError: 393614:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12559.640274] Lustre: server umount lustre-MDT0001 complete [12566.180559] Lustre: server umount lustre-OST0000 complete [12598.026527] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing set_hostid [12610.054454] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing load_modules_local [12646.077622] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12320 sec ========= 04:28:54 (1788164934) [12648.258293] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:28:56 (1788164936) === [12651.800124] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:28:59 (1788164939) === [12690.043555] Lustre: DEBUG MARKER: oleg625-server.virtnet: executing unload_modules_local [12692.779769] Key type lgssc unregistered [12693.155721] LNet: 400337:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12693.165684] LNetError: 400337:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12693.188178] LNet: Removed LNI 192.168.206.125@tcp [12694.087700] Key type .llcrypt unregistered [12694.090561] Key type ._llcrypt unregistered