[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 470958992 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001008] APIC: Switch to symmetric I/O mode setup [ 0.002223] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004017] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010128] LSM: Security Framework initializing [ 0.011054] Yama: becoming mindful. [ 0.012037] SELinux: Initializing. [ 0.014007] *** VALIDATE selinux *** [ 0.022153] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026753] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028147] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029106] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031114] *** VALIDATE tmpfs *** [ 0.032384] *** VALIDATE proc *** [ 0.034220] *** VALIDATE cgroup *** [ 0.035011] *** VALIDATE cgroup2 *** [ 0.037368] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038163] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040028] Spectre V2 : User space: Vulnerable [ 0.041010] Speculative Store Bypass: Vulnerable [ 0.044318] debug: unmapping init [mem 0xffffffff84059000-0xffffffff84060fff] [ 0.046163] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047536] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048023] ... version: 2 [ 0.049017] ... bit width: 48 [ 0.050017] ... generic registers: 4 [ 0.051015] ... value mask: 0000ffffffffffff [ 0.052017] ... max period: 00007fffffffffff [ 0.053017] ... fixed-purpose events: 3 [ 0.054018] ... event mask: 000000070000000f [ 0.056203] rcu: Hierarchical SRCU implementation. [ 0.058369] smp: Bringing up secondary CPUs ... [ 0.059633] x86: Booting SMP configuration: [ 0.060027] .... node #0, CPUs: #1 #2 #3 [ 0.063703] smp: Brought up 1 node, 4 CPUs [ 0.065015] smpboot: Max logical packages: 1 [ 0.066017] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.222182] node 0 deferred pages initialised in 154ms [ 0.225137] devtmpfs: initialized [ 0.226229] x86/mm: Memory block size: 128MB [ 0.229066] gcov: version magic: 0x41383552 [ 0.231282] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.232104] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.233288] pinctrl core: initialized pinctrl subsystem [ 0.234268] [ 0.234892] ************************************************************* [ 0.235014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.236010] ** ** [ 0.237017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.238014] ** ** [ 0.239021] ** This means that this kernel is built to expose internal ** [ 0.240016] ** IOMMU data structures, which may compromise security on ** [ 0.241022] ** your system. ** [ 0.242013] ** ** [ 0.243012] ** If you see this message and you are not debugging the ** [ 0.244013] ** kernel, report this immediately to your vendor! ** [ 0.245017] ** ** [ 0.246013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.247014] ************************************************************* [ 0.248658] NET: Registered protocol family 16 [ 0.249467] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.250073] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.251065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.252467] cpuidle: using governor menu [ 0.254777] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.257430] PCI: Using configuration type 1 for base access [ 0.260127] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.271096] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.272031] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.274052] cryptd: max_cpu_qlen set to 1000 [ 0.277076] ACPI: Added _OSI(Module Device) [ 0.278013] ACPI: Added _OSI(Processor Device) [ 0.279014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.280011] ACPI: Added _OSI(Processor Aggregator Device) [ 0.284713] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.286444] ACPI: Interpreter enabled [ 0.287070] ACPI: PM: (supports S0 S3 S4 S5) [ 0.288013] ACPI: Using IOAPIC for interrupt routing [ 0.289116] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.290398] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.299513] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.300043] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.301020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.302080] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.304314] acpiphp: Slot [2] registered [ 0.305103] acpiphp: Slot [5] registered [ 0.305904] acpiphp: Slot [6] registered [ 0.306065] acpiphp: Slot [7] registered [ 0.307074] acpiphp: Slot [8] registered [ 0.307971] acpiphp: Slot [9] registered [ 0.308089] acpiphp: Slot [10] registered [ 0.309151] acpiphp: Slot [3] registered [ 0.310104] acpiphp: Slot [4] registered [ 0.311104] acpiphp: Slot [11] registered [ 0.312110] acpiphp: Slot [12] registered [ 0.313104] acpiphp: Slot [13] registered [ 0.314122] acpiphp: Slot [14] registered [ 0.315104] acpiphp: Slot [15] registered [ 0.316140] acpiphp: Slot [16] registered [ 0.318143] acpiphp: Slot [17] registered [ 0.320117] acpiphp: Slot [18] registered [ 0.321100] acpiphp: Slot [19] registered [ 0.322161] acpiphp: Slot [20] registered [ 0.324100] acpiphp: Slot [21] registered [ 0.325182] acpiphp: Slot [22] registered [ 0.327098] acpiphp: Slot [23] registered [ 0.328101] acpiphp: Slot [24] registered [ 0.330097] acpiphp: Slot [25] registered [ 0.331097] acpiphp: Slot [26] registered [ 0.332081] acpiphp: Slot [27] registered [ 0.334107] acpiphp: Slot [28] registered [ 0.335095] acpiphp: Slot [29] registered [ 0.337091] acpiphp: Slot [30] registered [ 0.338132] acpiphp: Slot [31] registered [ 0.339085] PCI host bridge to bus 0000:00 [ 0.341019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.343019] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.345024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.347028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.349018] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.352021] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.353144] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.356876] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.360309] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.369826] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.374834] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.378018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.380017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.382013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.385546] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.388791] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.391060] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.395538] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.401016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.414015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.418012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.423007] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.430016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.437015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.457013] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.466479] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.474017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.486015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.504016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.516150] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.524071] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.532017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.555015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.568129] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.575015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.581015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.601013] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.615276] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.626014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.639014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.658016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.669918] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.681015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.688016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.709172] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.719935] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.724195] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.728382] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.730359] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.733235] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.738074] iommu: Default domain type: Passthrough [ 0.741395] SCSI subsystem initialized [ 0.743162] ACPI: bus type USB registered [ 0.745127] usbcore: registered new interface driver usbfs [ 0.748098] usbcore: registered new interface driver hub [ 0.750075] usbcore: registered new device driver usb [ 0.752171] pps_core: LinuxPPS API ver. 1 registered [ 0.753008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.756057] PTP clock support registered [ 0.759090] EDAC MC: Ver: 3.0.0 [ 0.761160] PCI: Using ACPI for IRQ routing [ 0.763515] NetLabel: Initializing [ 0.765015] NetLabel: domain hash size = 128 [ 0.767013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.769079] NetLabel: unlabeled traffic allowed by default [ 0.771371] vgaarb: loaded [ 0.773222] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.774010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.781000] clocksource: Switched to clocksource kvm-clock [ 0.879916] VFS: Disk quotas dquot_6.6.0 [ 0.881515] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.884552] *** VALIDATE ramfs *** [ 0.886248] *** VALIDATE hugetlbfs *** [ 0.888482] pnp: PnP ACPI init [ 0.891370] pnp: PnP ACPI: found 6 devices [ 0.909901] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.914331] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.917110] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.919594] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.922347] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.925276] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.928566] NET: Registered protocol family 2 [ 0.931162] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.936412] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.939475] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.946598] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.950497] TCP: Hash tables configured (established 65536 bind 65536) [ 0.955371] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.961503] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.965469] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.969309] NET: Registered protocol family 1 [ 0.972173] RPC: Registered named UNIX socket transport module. [ 0.974611] RPC: Registered udp transport module. [ 0.976191] RPC: Registered tcp transport module. [ 0.978163] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.981035] NET: Registered protocol family 44 [ 0.982726] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.985216] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.987496] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.990044] PCI: CLS 0 bytes, default 64 [ 0.991835] Unpacking initramfs... [ 2.407530] debug: unmapping init [mem 0xffff94b2fcc54000-0xffff94b2fffbffff] [ 2.411939] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.413892] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.417217] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.910709] Initialise system trusted keyrings [ 2.912206] Key type blacklist registered [ 2.914108] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.922267] zbud: loaded [ 2.924978] *** VALIDATE nfs *** [ 2.926168] *** VALIDATE nfs4 *** [ 2.927831] pstore: using deflate compression [ 2.931576] Platform Keyring initialized [ 3.031874] NET: Registered protocol family 38 [ 3.033519] Key type asymmetric registered [ 3.035588] Asymmetric key parser 'x509' registered [ 3.037603] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.041080] io scheduler mq-deadline registered [ 3.043069] io scheduler kyber registered [ 3.044445] io scheduler bfq registered [ 3.046634] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.049714] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.051952] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.054270] ACPI: Power Button [PWRF] [ 3.059558] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.064599] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.079667] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.086995] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.104945] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.130839] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.159094] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.163797] Non-volatile memory driver v1.3 [ 3.165712] Linux agpgart interface v0.103 [ 3.197215] virtio_blk virtio1: [vda] 145784 512-byte logical blocks (74.6 MB/71.2 MiB) [ 3.200612] vda: detected capacity change from 0 to 74641408 [ 3.215101] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.218345] vdb: detected capacity change from 0 to 1073741824 [ 3.241219] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.243964] vdc: detected capacity change from 0 to 2621440000 [ 3.262264] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.265576] vdd: detected capacity change from 0 to 2621440000 [ 3.282869] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.285663] vde: detected capacity change from 0 to 4294967296 [ 3.299747] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.302547] vdf: detected capacity change from 0 to 4294967296 [ 3.310642] libphy: Fixed MDIO Bus: probed [ 3.315874] usbcore: registered new interface driver usbserial_generic [ 3.318732] usbserial: USB Serial support registered for generic [ 3.321296] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.326220] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.328247] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.330893] mousedev: PS/2 mouse device common for all mice [ 3.334284] rtc_cmos 00:05: RTC can wake from S4 [ 3.337972] rtc_cmos 00:05: registered as rtc0 [ 3.340483] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.344900] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.348512] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.349278] intel_pstate: CPU model not supported [ 3.354456] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.361709] hid: raw HID events driver (C) Jiri Kosina [ 3.363939] usbcore: registered new interface driver usbhid [ 3.367111] usbhid: USB HID core driver [ 3.369349] drop_monitor: Initializing network drop monitor service [ 3.372283] Initializing XFRM netlink socket [ 3.374650] NET: Registered protocol family 10 [ 3.377941] Segment Routing with IPv6 [ 3.379059] NET: Registered protocol family 17 [ 3.380564] mpls_gso: MPLS GSO support [ 3.384310] RAS: Correctable Errors collector initialized. [ 3.386189] AVX version of gcm_enc/dec engaged. [ 3.387949] AES CTR mode by8 optimization enabled [ 3.471856] sched_clock: Marking stable (3471826662, 0)->(4403427394, -931600732) [ 3.476196] registered taskstats version 1 [ 3.478733] Loading compiled-in X.509 certificates [ 3.480665] zswap: loaded using pool lzo/zbud [ 3.505903] Key type big_key registered [ 3.518175] Key type encrypted registered [ 3.522138] ima: No TPM chip found, activating TPM-bypass! [ 3.526177] ima: Allocated hash algorithm: sha1 [ 3.529575] ima: No architecture policies found [ 3.532595] evm: Initialising EVM extended attributes: [ 3.536236] evm: security.selinux [ 3.538289] evm: security.ima [ 3.540295] evm: security.capability [ 3.542707] evm: HMAC attrs: 0x1 [ 3.546268] rtc_cmos 00:05: setting system clock to 2026-07-18 19:29:57 UTC (1784402997) [ 3.555532] debug: unmapping init [mem 0xffffffff85003000-0xffffffff851fffff] [ 3.561191] debug: unmapping init [mem 0xffffffff83d82000-0xffffffff84058fff] [ 3.570336] Write protecting the kernel read-only data: 28672k [ 3.574884] debug: unmapping init [mem 0xffffffff82403000-0xffffffff825fffff] [ 3.578017] debug: unmapping init [mem 0xffffffff82d14000-0xffffffff82dfffff] [ 3.610514] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.620081] systemd[1]: Detected virtualization kvm. [ 3.622174] systemd[1]: Detected architecture x86-64. [ 3.624396] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.652278] systemd[1]: No hostname configured. [ 3.654279] systemd[1]: Set hostname to . [ 3.656618] random: systemd: uninitialized urandom read (16 bytes read) [ 3.659703] systemd[1]: Initializing machine ID from random generator. [ 3.703071] random: ln: uninitialized urandom read (6 bytes read) [ 3.799166] random: systemd: uninitialized urandom read (16 bytes read) [ 3.801762] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.810065] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.818556] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.483096] device-mapper: uevent: version 1.0.3 [ 4.485142] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.192158] virtio_net virtio0 ens2: renamed from eth0 [ 5.368884] random: fast init done [ 5.421182] scsi host0: ata_piix [ 5.425665] scsi host1: ata_piix [ 5.428383] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.432427] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.989193] dracut-initqueue[584]: RTNETLINK answers: File exists [ 10.051148] random: crng init done [ 10.053255] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.475988] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.734788] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.001191] SELinux: Disabled at runtime. [ 12.066146] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.075393] systemd[1]: Detected virtualization kvm. [ 12.077408] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.587301] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.592310] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.598356] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.603598] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.609170] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.618948] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.628133] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. Starting Create list of required st…ce nodes for the current kernel... Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting POSIX Message Queue File System... [ OK ] Reached target RPC Port Mapper. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ 12.779735] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Mounting Huge Pages File System... [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.174817] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.641125] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.643119] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.744367] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.756169] EDAC sbridge: Ver: 1.1.2 [ 15.359120] Key type dns_resolver registered [ 15.662881] NFS: Registering the id_resolver key type [ 15.666070] Key type id_resolver registered [ 15.667684] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... Starting Network Manager... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg317-server login: [ 42.098911] libcfs: loading out-of-tree module taints kernel. [ 42.118975] Key type ._llcrypt registered [ 42.120760] Key type .llcrypt registered [ 42.170964] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 53.551422] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 54.809930] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 54.831247] alg: No test for adler32 (adler32-zlib) [ 56.172215] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 56.891517] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 58.599167] Key type lgssc registered [ 60.005477] Lustre: Echo OBD driver; http://www.lustre.org/ [ 76.980851] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 81.510160] hrtimer: interrupt took 4457129 ns [ 115.770716] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 128.050907] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 128.083300] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 129.457164] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 129.499195] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 129.574810] Lustre: lustre-MDT0000: new disk, initializing [ 129.697214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 129.713026] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 133.621045] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 145.807337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 145.892915] Lustre: 6490:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 145.917953] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 145.920896] Lustre: Skipped 1 previous similar message [ 145.989563] Lustre: lustre-MDT0001: new disk, initializing [ 146.060139] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 146.104922] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 146.121872] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 150.555220] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 155.640702] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 165.793255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 166.108303] Lustre: lustre-OST0000: new disk, initializing [ 166.121075] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 166.127795] Lustre: 8426:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 166.224061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 172.338237] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 174.632129] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 174.645121] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 174.722870] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 186.760862] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 186.898306] Lustre: lustre-OST0001: new disk, initializing [ 186.900804] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 186.908502] Lustre: 9496:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 186.957898] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 192.868606] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 197.162232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 197.173737] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 197.262559] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 206.202496] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 213.751852] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 222.151026] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing check_logdir /tmp/testlogs/ [ 227.182408] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing yml_node [ 232.005454] Lustre: DEBUG MARKER: Client: 2.17.54.174 [ 234.605165] Lustre: DEBUG MARKER: MDS: 2.17.54.174 [ 237.044770] Lustre: DEBUG MARKER: OSS: 2.17.54.174 [ 238.547642] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Jul 18 15:33:49 EDT 2026 [ 255.027047] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 256.516442] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 273.889270] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 273.890352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 273.908412] Lustre: Skipped 3 previous similar messages [ 273.913691] Lustre: Skipped 2 previous similar messages [ 279.008620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 279.018084] Lustre: Skipped 4 previous similar messages [ 284.129455] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 284.143346] Lustre: Skipped 3 previous similar messages [ 286.175135] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 286.378922] Lustre: server umount lustre-MDT0000 complete [ 289.661627] LustreError: 6481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784403283 with bad export cookie 17082381063234009404 [ 289.668277] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 289.674920] LustreError: 6481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 294.372846] LustreError: 10020:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 294.374024] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 294.380458] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 294.391511] LustreError: 10020:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 294.421221] Lustre: Skipped 1 previous similar message [ 295.976791] Lustre: server umount lustre-MDT0001 complete [ 299.931470] Lustre: server umount lustre-OST0000 complete [ 303.641104] Lustre: server umount lustre-OST0001 complete [ 309.898934] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 318.935458] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 351.103235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 351.347781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 351.391425] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 351.554190] Lustre: lustre-MDT0000: new disk, initializing [ 351.658818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 351.670183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 355.414406] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 364.836361] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 364.942333] Lustre: 15048:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 365.013358] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 365.016442] Lustre: Skipped 1 previous similar message [ 365.127481] Lustre: lustre-MDT0001: new disk, initializing [ 365.215147] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 365.247849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 365.258457] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 369.062301] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 377.161823] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 381.938248] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 388.183832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 388.391553] Lustre: lustre-OST0000: new disk, initializing [ 388.394186] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 388.399709] Lustre: 17018:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 388.481036] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 390.254115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 390.271960] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 390.449067] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 394.567568] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 403.806713] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 408.518778] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 408.688947] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 413.599723] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 413.816162] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 416.235673] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 416.243897] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 416.249253] LustreError: Skipped 1 previous similar message [ 416.254377] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 416.274455] Lustre: Skipped 1 previous similar message [ 416.276557] Lustre: Skipped 1 previous similar message [ 421.963576] Lustre: server umount lustre-OST0000 complete [ 426.465224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 426.495446] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 426.503482] Lustre: Skipped 1 previous similar message [ 426.535265] Lustre: Skipped 4 previous similar messages [ 440.287128] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 440.461678] Lustre: server umount lustre-MDT0000 complete [ 441.833444] LustreError: 15950:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 441.866954] LustreError: 15950:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 444.030685] LustreError: 15042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784403437 with bad export cookie 17082381063234010930 [ 444.037685] LustreError: 15042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 444.038459] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 444.381569] Lustre: server umount lustre-MDT0001 complete [ 451.588400] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 15:37:22 (1784403442) [ 458.765475] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 459.534476] LustreError: 19283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 459.653188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 464.632656] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 464.867128] LustreError: 19284:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 469.987691] LustreError: 19283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 474.659130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.920822] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 488.581555] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 495.087683] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 502.697138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 508.852252] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 517.682207] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 524.831438] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 536.035043] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 536.042081] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 536.051897] Lustre: Skipped 1 previous similar message [ 536.068353] Lustre: Skipped 6 previous similar messages [ 538.473926] Lustre: server umount lustre-MDT0000 complete [ 541.164862] LustreError: 19283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.190069] LustreError: 19283:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 542.506236] LustreError: 19265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784403536 with bad export cookie 17082381063234011819 [ 542.517743] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 542.524462] LustreError: 19265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 542.978348] Lustre: server umount lustre-MDT0001 complete [ 556.996722] Lustre: server umount lustre-OST0000 complete [ 578.637197] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 588.837191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 589.195491] LustreError: 23665:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 589.280926] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 589.295325] Lustre: Skipped 2 previous similar messages [ 593.177980] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 601.303084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 606.587896] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 609.691150] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 617.189299] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 624.347274] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 633.527122] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 633.764717] Lustre: lustre-OST0001: new disk, initializing [ 633.769945] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 633.778692] Lustre: 25980:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 635.215795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 635.233165] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 635.354114] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 640.330426] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 650.770393] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 655.512466] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 671.201102] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 671.206781] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 671.212233] Lustre: Skipped 4 previous similar messages [ 671.221125] Lustre: Skipped 3 previous similar messages [ 675.401935] Lustre: server umount lustre-MDT0000 complete [ 676.326895] LustreError: 23659:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 676.346142] LustreError: 23659:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 679.020920] LustreError: 23645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784403672 with bad export cookie 17082381063234012666 [ 679.024687] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 679.034185] LustreError: 23645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 679.311903] Lustre: server umount lustre-MDT0001 complete [ 692.249523] Lustre: server umount lustre-OST0000 complete [ 705.889791] Lustre: server umount lustre-OST0001 complete [ 722.199273] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 732.617145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 733.201324] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 733.207243] Lustre: Skipped 3 previous similar messages [ 738.014965] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 743.393751] LustreError: 29004:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 743.410771] LustreError: 29004:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 746.344516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 751.513778] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 754.527727] Lustre: 30151:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 760.976626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 767.091053] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 775.378399] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 781.674809] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 790.150685] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 794.447291] Lustre: 32031:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 803.297773] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 803.314529] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 803.333274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 807.667698] Lustre: server umount lustre-MDT0000 complete [ 811.538844] LustreError: 30532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784403805 with bad export cookie 17082381063234013954 [ 811.559800] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 811.957972] Lustre: server umount lustre-MDT0001 complete [ 825.335821] Lustre: server umount lustre-OST0000 complete [ 839.685744] Lustre: server umount lustre-OST0001 complete [ 847.648691] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 15:43:59 (1784403839) [ 875.262751] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 885.294762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 885.631370] LustreError: 34830:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 885.650834] LustreError: 34830:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 889.741541] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 897.943470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 902.054599] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 904.588487] Lustre: 35981:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 910.597709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 916.725459] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 924.973202] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 931.860758] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 939.314814] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 942.431229] Lustre: 37858:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 949.730206] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 949.744268] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 949.754188] Lustre: Skipped 3 previous similar messages [ 949.758153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 949.763640] Lustre: Skipped 4 previous similar messages [ 953.186868] Lustre: server umount lustre-MDT0000 complete [ 957.069605] LustreError: 34812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784403951 with bad export cookie 17082381063234015060 [ 957.072424] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 957.080462] LustreError: 34812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 957.433302] Lustre: server umount lustre-MDT0001 complete [ 971.568323] Lustre: server umount lustre-OST0000 complete [ 985.834547] Lustre: server umount lustre-OST0001 complete [ 995.887290] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 15:46:26 (1784403986) [ 1015.641970] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1027.255368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1027.906904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1027.912909] Lustre: Skipped 7 previous similar messages [ 1033.392635] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1043.268830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1048.841663] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1052.252646] Lustre: 41155:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1059.449106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1065.547992] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1075.820615] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1083.373143] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1091.323785] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1095.336359] Lustre: 43032:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1096.300581] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1096.307338] Lustre: Skipped 1 previous similar message [ 1100.256328] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1100.261662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1100.271351] Lustre: Skipped 3 previous similar messages [ 1106.146624] Lustre: server umount lustre-MDT0000 complete [ 1110.215808] LustreError: 41157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784404104 with bad export cookie 17082381063234016173 [ 1110.219827] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1110.234690] LustreError: 41157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1116.858862] Lustre: server umount lustre-MDT0001 complete [ 1126.246289] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1135.586241] LustreError: 3621:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff94b381b02a00 x1871082212955904/t0(0) o250->MGC192.168.203.117@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1140.536690] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1145.267707] LustreError: 41519:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1145.276704] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1145.291036] LustreError: 41519:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 35 previous similar messages [ 1149.689718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1155.157268] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1155.575915] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1155.585862] Lustre: Skipped 1 previous similar message [ 1163.402242] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1165.327584] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1171.972840] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1173.861825] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1180.641218] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1186.864922] Lustre: server umount lustre-MDT0000 complete [ 1191.224072] LustreError: 41157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784404185 with bad export cookie 17082381063234017300 [ 1191.227635] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1191.238049] LustreError: 41157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1191.609100] Lustre: server umount lustre-MDT0001 complete [ 1198.135861] Lustre: server umount lustre-OST0000 complete [ 1201.854569] Lustre: server umount lustre-OST0001 complete [ 1211.006329] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 15:50:02 (1784404202) [ 1226.791547] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1236.995942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1242.683741] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1253.040663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1259.220075] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1262.138467] Lustre: 48793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1269.296938] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1277.442205] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1287.286268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1294.444464] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1302.402182] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1306.923855] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1320.142958] Lustre: server umount lustre-MDT0000 complete [ 1320.418592] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1320.426877] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1320.437095] Lustre: Skipped 11 previous similar messages [ 1323.928274] LustreError: 47619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784404317 with bad export cookie 17082381063234018091 [ 1323.935458] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1323.948195] LustreError: 47619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1324.357433] Lustre: server umount lustre-MDT0001 complete [ 1339.095918] Lustre: server umount lustre-OST0000 complete [ 1352.654339] Lustre: server umount lustre-OST0001 complete [ 1365.515835] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 15:52:36 (1784404356) [ 1368.005958] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1370.057115] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 15:52:41 (1784404361) [ 1379.711363] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 1389.228639] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1412.395734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1412.652834] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1412.759407] Lustre: lustre-MDT0000: new disk, initializing [ 1412.888318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1417.959382] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1430.606196] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1439.021708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1439.265468] Lustre: lustre-OST0000: new disk, initializing [ 1439.270075] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1439.273118] Lustre: Skipped 1 previous similar message [ 1439.278382] Lustre: 54875:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1441.103569] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1441.124397] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1441.166844] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1445.616679] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1454.850833] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1457.150271] Lustre: DEBUG MARKER: create test files [ 1492.451088] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1492.467285] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1492.475063] Lustre: Skipped 26 previous similar messages [ 1495.722331] Lustre: server umount lustre-OST0000 complete [ 1498.898394] Lustre: server umount lustre-MDT0000 complete [ 1510.621828] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 1513.316847] Key type lgssc unregistered [ 1513.602876] LNet: 56506:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1513.624936] LNetError: 56506:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1513.651892] LNet: Removed LNI 192.168.203.117@tcp [ 1514.734157] Key type .llcrypt unregistered [ 1514.738671] Key type ._llcrypt unregistered [ 1540.182793] Key type ._llcrypt registered [ 1540.185132] Key type .llcrypt registered [ 1540.308980] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1541.292558] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1541.357283] alg: No test for adler32 (adler32-zlib) [ 1542.356275] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 1542.613685] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 1544.263382] Key type lgssc registered [ 1545.257821] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1551.575548] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1551.603666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1552.997120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1557.007378] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1563.924456] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1569.619249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1569.891681] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1575.566681] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1577.069097] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1577.154763] Lustre: 59240:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1577.162631] Lustre: 59240:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1577.168870] Lustre: 59240:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1577.179459] Lustre: 59240:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1577.190982] Lustre: 59240:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1577.199033] Lustre: 59240:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1581.848845] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1586.587348] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1592.140388] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1628.641754] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1628.645396] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1628.665089] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1629.804442] Lustre: server umount lustre-OST0000 complete [ 1632.972490] Lustre: server umount lustre-MDT0000 complete [ 1641.678669] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 1643.767393] Key type lgssc unregistered [ 1643.986947] LNet: 60874:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1643.995636] LNetError: 60874:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1644.017824] LNet: Removed LNI 192.168.203.117@tcp [ 1644.749150] Key type .llcrypt unregistered [ 1644.751184] Key type ._llcrypt unregistered [ 1671.974279] Key type ._llcrypt registered [ 1671.980983] Key type .llcrypt registered [ 1672.084412] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1673.362887] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1673.418493] alg: No test for adler32 (adler32-zlib) [ 1674.383328] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 1674.559771] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 1676.191199] Key type lgssc registered [ 1677.134609] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1683.471378] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1683.494369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1684.812739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1688.476347] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1694.551978] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1699.962179] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1700.264277] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1704.779640] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1709.488862] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1709.538584] Lustre: 63715:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1709.545531] Lustre: 63715:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1709.550293] Lustre: 63715:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1709.554687] Lustre: 63715:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1709.559546] Lustre: 63715:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1709.563228] Lustre: 63715:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1710.726430] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1716.442952] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1724.897030] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1724.900033] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1724.907429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1729.754434] Lustre: server umount lustre-OST0000 complete [ 1732.699105] Lustre: server umount lustre-MDT0000 complete [ 1741.499055] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 1743.404096] Key type lgssc unregistered [ 1743.640342] LNet: 65272:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1743.645972] LNetError: 65272:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1743.660375] LNet: Removed LNI 192.168.203.117@tcp [ 1744.263212] Key type .llcrypt unregistered [ 1744.265442] Key type ._llcrypt unregistered [ 1756.568939] Key type ._llcrypt registered [ 1756.574074] Key type .llcrypt registered [ 1756.647444] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 1767.515813] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1768.069895] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1768.115490] alg: No test for adler32 (adler32-zlib) [ 1769.117681] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 1769.299976] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 1770.911309] Key type lgssc registered [ 1771.736920] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1799.777048] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1799.795609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1801.000787] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1801.034297] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1801.116212] Lustre: lustre-MDT0000: new disk, initializing [ 1801.207964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1801.218914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1804.430496] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1811.519237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1811.595739] Lustre: 68403:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1811.663773] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1811.667123] Lustre: Skipped 1 previous similar message [ 1811.715364] Lustre: lustre-MDT0001: new disk, initializing [ 1811.773677] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1811.799084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1811.805404] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1814.540523] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1821.436484] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1825.524063] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1830.090466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1830.345390] Lustre: lustre-OST0000: new disk, initializing [ 1830.351407] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1830.359504] Lustre: 70371:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1830.433590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1835.087552] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1837.615238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1837.630590] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1837.723939] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1842.608309] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1846.138681] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1846.338915] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1849.939916] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1850.113965] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1853.419195] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1853.420386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1853.458096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1853.461821] LustreError: Skipped 1 previous similar message [ 1853.488346] Lustre: Skipped 1 previous similar message [ 1857.780426] Lustre: server umount lustre-OST0000 complete [ 1863.136452] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1863.144628] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1863.158572] Lustre: Skipped 1 previous similar message [ 1863.167128] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1866.889427] Lustre: server umount lustre-MDT0000 complete [ 1870.487030] LustreError: 68396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784404864 with bad export cookie 16985740887409933299 [ 1870.493274] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1870.500817] LustreError: 68396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1870.846544] Lustre: server umount lustre-MDT0001 complete [ 1878.128373] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:01:09 (1784404869) [ 1886.887481] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1916.812977] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 1924.721090] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 1952.542081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1952.718117] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1952.739878] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1952.804705] Lustre: lustre-MDT0000: new disk, initializing [ 1952.874987] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1952.898747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1955.989711] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1963.833310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1963.946482] Lustre: 76400:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1963.993411] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1963.997215] Lustre: Skipped 1 previous similar message [ 1964.085434] Lustre: lustre-MDT0001: new disk, initializing [ 1964.157337] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1964.206542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1964.218779] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1967.548535] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1974.970573] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1979.994691] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1985.705739] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1985.918983] Lustre: lustre-OST0000: new disk, initializing [ 1985.921970] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1985.930240] Lustre: 78365:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1985.990635] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1991.002083] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 1992.738493] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1992.748777] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1992.843183] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1998.239366] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2001.750155] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2001.931535] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2005.375693] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2005.527379] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2008.047263] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2008.048425] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2008.060700] Lustre: Skipped 1 previous similar message [ 2008.061980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2008.084021] Lustre: Skipped 2 previous similar messages [ 2013.159043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2013.168354] Lustre: Skipped 1 previous similar message [ 2018.275145] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2021.343152] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2021.464569] Lustre: server umount lustre-OST0000 complete [ 2025.967554] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2025.974717] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2025.984448] Lustre: Skipped 1 previous similar message [ 2028.512245] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2028.515373] Lustre: Skipped 3 previous similar messages [ 2039.263375] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2039.436742] Lustre: server umount lustre-MDT0000 complete [ 2042.310545] LustreError: 76392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784405036 with bad export cookie 16985740887409934181 [ 2042.311594] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2042.316351] LustreError: 76392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2042.635360] Lustre: server umount lustre-MDT0001 complete [ 2049.060858] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:04:00 (1784405040) [ 2054.236281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2054.504820] LustreError: 80778:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2054.519636] LustreError: 80778:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2054.556780] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2057.592876] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2059.750057] LustreError: 80779:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2063.742773] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2066.917857] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2072.281970] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2075.884560] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2080.282072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2084.621855] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2090.085726] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2092.448590] Lustre: 83454:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 03b908dc-86e8-4a76-9451-9b3759f733cc at adminstrative request [ 2094.060220] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2094.070355] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2104.290599] Lustre: MGS: Client 03b908dc-86e8-4a76-9451-9b3759f733cc (at 0@lo) reconnecting [ 2104.298536] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xebb97957c0e5c2d7 to 0xebb97957c0e5c570 [ 2104.306650] Lustre: MGC192.168.203.117@tcp: Connection restored to 0@lo (at 0@lo) [ 2126.711420] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2126.850927] Lustre: lustre-OST0001: new disk, initializing [ 2126.853703] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2126.858380] Lustre: 83844:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2126.910835] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2126.914294] Lustre: Skipped 2 previous similar messages [ 2131.008759] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2135.659140] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2135.670691] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2135.787986] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2138.310441] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2141.155224] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2141.155820] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2141.158289] LustreError: Skipped 1 previous similar message [ 2141.165312] Lustre: Skipped 1 previous similar message [ 2141.170572] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2141.176176] Lustre: Skipped 5 previous similar messages [ 2142.488740] Lustre: server umount lustre-OST0001 complete [ 2146.281765] LustreError: 82599:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2146.294507] LustreError: 82599:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2151.109837] Lustre: server umount lustre-OST0000 complete [ 2156.002470] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2156.008765] LustreError: Skipped 2 previous similar messages [ 2156.012988] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2156.028150] Lustre: Skipped 2 previous similar messages [ 2160.324687] Lustre: server umount lustre-MDT0000 complete [ 2161.633348] LustreError: 80779:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2161.646141] LustreError: 80779:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2163.079164] LustreError: 81515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784405157 with bad export cookie 16985740887409935728 [ 2163.081844] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2163.089520] LustreError: 81515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2163.276735] Lustre: server umount lustre-MDT0001 complete [ 2169.896959] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:06:01 (1784405161) [ 2171.149172] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2172.406976] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:06:04 (1784405164) [ 2173.559595] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2174.913473] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:06:06 (1784405166) [ 2175.989661] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2177.372097] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:06:09 (1784405169) [ 2178.464829] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2179.728102] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:06:11 (1784405171) [ 2181.004056] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2185.446501] loop: module loaded [ 2185.466304] loop0: detected capacity change from 0 to 4294967296 [ 2185.500453] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2186.595859] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2189.058620] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2192.245106] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2211.679070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2211.730507] LustreError: 87964:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2211.742948] LustreError: 87964:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2211.748826] LustreError: 87964:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2211.756577] LustreError: 87964:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2224.873565] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 2230.456301] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 2255.784736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2255.987409] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2256.011722] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2256.072568] Lustre: lustre-MDT0000: new disk, initializing [ 2256.122042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2256.132904] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2259.508379] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2267.284851] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2267.378317] Lustre: 90828:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2267.578985] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2270.938463] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2277.490950] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2281.106048] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2285.031225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2285.217322] Lustre: 92795:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2288.878864] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2294.312353] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2294.367689] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2295.002561] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2297.451021] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2297.568286] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2299.902568] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2300.016482] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2304.481152] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2304.483261] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2304.486250] LustreError: Skipped 1 previous similar message [ 2304.494890] Lustre: Skipped 1 previous similar message [ 2304.497069] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2304.506572] Lustre: Skipped 7 previous similar messages [ 2307.176549] Lustre: server umount lustre-OST0000 complete [ 2315.466936] Lustre: server umount lustre-MDT0000 complete [ 2317.593396] LustreError: 90820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784405311 with bad export cookie 16985740887409936351 [ 2317.598100] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2317.598995] LustreError: 90820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2317.812175] Lustre: server umount lustre-MDT0001 complete [ 2322.831441] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:08:34 (1784405314) [ 2326.921278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2327.211069] LustreError: 95203:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2329.882611] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2335.178028] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2338.074581] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2342.741499] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2345.840252] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2350.038058] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2353.397707] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2357.839265] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2361.087824] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2361.232568] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2364.119728] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2364.246751] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2368.809728] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2368.940960] Lustre: lustre-OST0001: new disk, initializing [ 2368.944449] Lustre: Skipped 2 previous similar messages [ 2368.947661] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2368.953784] Lustre: Skipped 3 previous similar messages [ 2368.959526] Lustre: 98568:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2372.996838] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2378.830257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2378.835132] Lustre: Skipped 2 previous similar messages [ 2378.839119] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2378.895114] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2379.196160] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2381.567668] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2381.694228] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2384.184433] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2384.335470] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2398.591872] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2418.129830] Lustre: Failing over lustre-MDT0000 [ 2418.443689] Lustre: server umount lustre-MDT0000 complete [ 2420.198591] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2420.199951] LustreError: 95200:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2420.215418] Lustre: Skipped 5 previous similar messages [ 2420.223593] LustreError: 95200:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2436.577497] Lustre: 66626:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784405414/real 1784405414] req@00000000a1b7e6e9 x1871084008998656/t0(0) o400->MGC192.168.203.117@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1784405430 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2436.597099] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2436.614906] LustreError: 96645:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2436.630324] LustreError: 96645:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 21 previous similar messages [ 2438.498692] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2438.500867] LDISKFS-fs (dm-0): recovery complete [ 2438.506370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2446.829414] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xebb97957c0e68fdd [ 2446.837556] Lustre: MGC192.168.203.117@tcp: Connection restored to 0@lo (at 0@lo) [ 2447.006549] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2447.009330] Lustre: Skipped 6 previous similar messages [ 2447.044898] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2447.643860] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2447.644137] Lustre: 100019:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2449.783360] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2452.454414] Lustre: 95930:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2452.467104] Lustre: 95930:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 2452.474974] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2452.529371] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2452.855105] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2453.179858] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2453.189503] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2453.506980] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2453.516668] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2453.521944] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2453.526770] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 2454.467466] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2454.475216] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 2454.791199] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2454.797276] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 2455.762318] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2455.777633] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 2456.719975] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2456.730478] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 2457.047130] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2457.051510] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 2459.904748] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2459.914351] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 2460.883984] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2460.895942] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2461.215095] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2461.221956] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2468.152199] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2468.164915] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 2469.122514] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2469.129605] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 2469.447149] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2469.453074] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 2484.231148] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2484.240577] Lustre: 101417:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 2485.184331] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2485.192035] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2485.511109] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2485.516092] LustreError: 101417:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2507.199788] Lustre: 101417:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2507.204356] Lustre: 101417:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2507.216050] Lustre: 101417:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 45459a00-b970-42a5-9484-cc2a8da81f61@192.168.203.17@tcp [ 2507.228374] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2507.235640] Lustre: 101417:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000dd563f8d x1871083996251008/t0(8589934770) o35->45459a00-b970-42a5-9484-cc2a8da81f61@192.168.203.17@tcp:6/0 lens 392/0 e 0 to 0 dl 1784405511 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2507.254662] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2507.271684] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2507.274789] Lustre: lustre-MDT0000: Denying connection for new client 45459a00-b970-42a5-9484-cc2a8da81f61 (at 192.168.203.17@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2507.297814] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2507.301994] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2507.305395] Lustre: Skipped 2 previous similar messages [ 2507.332028] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2507.332942] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2517.991203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2517.997765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2518.007735] Lustre: Skipped 1 previous similar message [ 2518.012941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2518.018376] Lustre: Skipped 4 previous similar messages [ 2522.298531] Lustre: server umount lustre-OST0000 complete [ 2524.133587] LustreError: 101785:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2524.147090] LustreError: 101785:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 13 previous similar messages [ 2530.822443] Lustre: server umount lustre-OST0001 complete [ 2539.421340] Lustre: server umount lustre-MDT0000 complete [ 2541.825665] LustreError: 97043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784405535 with bad export cookie 16985740887409987549 [ 2541.828186] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2541.835720] LustreError: 97043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2542.057323] Lustre: server umount lustre-MDT0001 complete [ 2547.608438] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:12:19 (1784405539) [ 2551.931973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2551.936374] Lustre: *** cfs_fail_loc=197, val=0*** [ 2567.391581] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2572.511476] LustreError: 103388:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.117@tcp: failed processing log, type 4: rc = -110 [ 2577.644745] Lustre: *** cfs_fail_loc=197, val=0*** [ 2577.646240] Lustre: Skipped 65 previous similar messages [ 2601.656403] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2605.957368] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2607.126739] Lustre: server umount lustre-OST0000 complete [ 2612.180605] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:13:24 (1784405604) [ 2628.742251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2629.012759] LustreError: 105006:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2629.026097] LustreError: 105006:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2631.484409] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2637.251233] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2639.869380] LustreError: 105037:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2639.875619] LustreError: 105037:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2639.881780] LustreError: 105037:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -5 [ 2640.047170] Lustre: server umount lustre-MDT0000 complete [ 2657.374347] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 2663.385366] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 2684.031227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2684.195477] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2684.219761] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2684.272381] Lustre: lustre-MDT0000: new disk, initializing [ 2684.327090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2686.726146] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2693.321770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2693.385613] Lustre: 109202:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2693.548639] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2695.655633] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2700.993917] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2703.983154] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2707.677344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2707.847877] Lustre: 111171:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2707.895652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2707.900768] Lustre: Skipped 4 previous similar messages [ 2708.576526] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2711.069455] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2715.973254] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2718.015603] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2718.128338] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2720.080399] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2720.174327] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2723.810453] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2723.810872] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2723.814015] LustreError: Skipped 3 previous similar messages [ 2723.815122] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2723.826810] Lustre: Skipped 6 previous similar messages [ 2723.842140] Lustre: Skipped 9 previous similar messages [ 2727.150360] Lustre: server umount lustre-OST0000 complete [ 2735.275911] Lustre: server umount lustre-MDT0000 complete [ 2737.025226] LustreError: 110009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784405730 with bad export cookie 16985740887409998147 [ 2737.028451] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2737.030589] LustreError: 110009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2737.164654] Lustre: server umount lustre-MDT0001 complete [ 2741.723479] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:15:33 (1784405733) [ 2746.319697] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 2747.732254] Key type lgssc unregistered [ 2747.918896] LNet: 113655:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2747.928236] LNetError: 113655:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2747.943484] LNet: Removed LNI 192.168.203.117@tcp [ 2748.428598] Key type .llcrypt unregistered [ 2748.431433] Key type ._llcrypt unregistered [ 2757.848031] Key type ._llcrypt registered [ 2757.850127] Key type .llcrypt registered [ 2757.905302] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 2766.329801] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 2766.734226] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2766.776238] alg: No test for adler32 (adler32-zlib) [ 2767.691617] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 2767.816087] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 2769.415159] Key type lgssc registered [ 2770.017489] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2790.403310] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2790.436322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2791.565663] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2791.584395] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2791.629810] Lustre: lustre-MDT0000: new disk, initializing [ 2791.697657] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2791.709284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2793.939766] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2799.903815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2800.080252] Lustre: lustre-OST0000: new disk, initializing [ 2800.086523] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2800.091576] Lustre: Skipped 1 previous similar message [ 2800.095430] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 2800.096581] Lustre: 117803:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2800.146134] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2803.568729] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2805.252939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2805.265253] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2805.289410] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2809.929996] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2810.026876] Lustre: lustre-OST0001: new disk, initializing [ 2810.029536] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2810.033752] Lustre: 118664:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2810.078043] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2813.343501] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2814.173771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2814.181569] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2814.201332] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2819.553084] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2819.555670] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2819.559340] Lustre: Skipped 1 previous similar message [ 2824.672765] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2824.678057] Lustre: Skipped 2 previous similar messages [ 2824.974741] Lustre: server umount lustre-MDT0000 complete [ 2828.487113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2831.244151] LustreError: 116990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784405825 with bad export cookie 14868640898668641907 [ 2831.248517] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2831.255819] LustreError: 116990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2831.304493] Lustre: server umount lustre-OST0000 complete [ 2833.247063] Lustre: server umount lustre-OST0001 complete [ 2837.498423] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 16:17:09 (1784405829) [ 2848.543526] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 16:17:20 (1784405840) [ 2853.087926] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 2863.773688] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 16:17:35 (1784405855) [ 2881.931619] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 2886.765412] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 2904.224145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2904.332119] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2904.345217] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2904.384809] Lustre: lustre-MDT0000: new disk, initializing [ 2904.415351] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2904.422265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2906.106437] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2911.249799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2911.302420] Lustre: 126402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2911.334375] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2911.337403] Lustre: Skipped 1 previous similar message [ 2911.380627] Lustre: lustre-MDT0001: new disk, initializing [ 2911.425228] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2911.444972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2911.451326] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2913.284285] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2917.947653] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2920.307622] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2923.075686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2923.197112] Lustre: lustre-OST0000: new disk, initializing [ 2923.199341] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2923.203165] Lustre: 128370:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2923.238454] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2925.681500] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2929.157075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2929.162046] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2929.204160] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2930.151777] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2931.977122] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2932.081405] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2933.941907] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2934.032250] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2939.360567] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2939.363947] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2939.366746] LustreError: Skipped 1 previous similar message [ 2939.373870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2939.380648] Lustre: Skipped 1 previous similar message [ 2941.025636] Lustre: server umount lustre-OST0000 complete [ 2944.481582] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2944.483393] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2944.491465] Lustre: Skipped 2 previous similar messages [ 2944.496869] Lustre: Skipped 1 previous similar message [ 2948.788862] Lustre: server umount lustre-MDT0000 complete [ 2950.379884] LustreError: 126397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784405944 with bad export cookie 14868640898668643132 [ 2950.382860] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2950.387538] LustreError: 126397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2950.524756] Lustre: server umount lustre-MDT0001 complete [ 2954.190995] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 16:19:06 (1784405946) [ 2957.198963] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2957.428510] LustreError: 130779:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2957.441676] LustreError: 130779:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2957.472494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2959.302803] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2962.912877] LustreError: 130780:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2963.495217] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2965.544502] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2969.121968] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2971.837443] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2974.942935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2977.440277] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 2980.555635] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2992.012627] Lustre: *** cfs_fail_loc=159, val=0*** [ 3007.281679] Lustre: lustre-MDT0000: Client 59f04179-99d7-4a3f-bd3c-87a33e8353a9 (at 192.168.203.17@tcp) reconnecting [ 3008.026742] Lustre: *** cfs_fail_loc=159, val=0*** [ 3008.030635] Lustre: Skipped 5 previous similar messages [ 3023.665219] Lustre: lustre-MDT0000: Client 59f04179-99d7-4a3f-bd3c-87a33e8353a9 (at 192.168.203.17@tcp) reconnecting [ 3043.807951] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3043.812352] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3043.822342] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3044.832204] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3045.978996] Lustre: server umount lustre-OST0000 complete [ 3049.953616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3049.958222] Lustre: Skipped 2 previous similar messages [ 3053.661680] Lustre: server umount lustre-MDT0000 complete [ 3055.072475] LustreError: 133333:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3055.080828] LustreError: 133333:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3055.156246] LustreError: 132619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784406049 with bad export cookie 14868640898668644014 [ 3055.160140] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3055.163458] LustreError: 132619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3055.293717] Lustre: server umount lustre-MDT0001 complete [ 3059.698658] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 3060.915239] Key type lgssc unregistered [ 3061.057367] LNet: 134570:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3061.060939] LNetError: 134570:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3061.068366] LNet: Removed LNI 192.168.203.117@tcp [ 3061.442754] Key type .llcrypt unregistered [ 3061.444981] Key type ._llcrypt unregistered [ 3072.816501] Key type ._llcrypt registered [ 3072.817588] Key type .llcrypt registered [ 3072.857140] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3073.209076] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3073.257115] alg: No test for adler32 (adler32-zlib) [ 3074.146460] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 3074.253766] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 3075.847231] Key type lgssc registered [ 3076.397558] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3079.806728] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3079.815964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3080.993666] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3082.635619] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3086.196295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3086.383569] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3088.052292] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3091.028526] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3093.280323] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3096.185522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3096.331897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3098.789593] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3102.042161] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3105.576918] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3112.457561] Lustre: *** cfs_fail_loc=159, val=0*** [ 3128.625651] Lustre: lustre-MDT0000: Client 4fff10d5-aa8a-45f0-aab2-eebba919a870 (at 192.168.203.17@tcp) reconnecting [ 3129.658467] Lustre: *** cfs_fail_loc=159, val=0*** [ 3146.041275] Lustre: lustre-MDT0001: Client 4fff10d5-aa8a-45f0-aab2-eebba919a870 (at 192.168.203.17@tcp) reconnecting [ 3146.736741] Lustre: *** cfs_fail_loc=159, val=0*** [ 3146.738943] Lustre: Skipped 3 previous similar messages [ 3162.928109] Lustre: lustre-MDT0001: Client 4fff10d5-aa8a-45f0-aab2-eebba919a870 (at 192.168.203.17@tcp) reconnecting [ 3164.551134] Lustre: 138995:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3165.588479] Lustre: *** cfs_fail_loc=159, val=0*** [ 3165.591059] Lustre: Skipped 4 previous similar messages [ 3181.872227] Lustre: lustre-MDT0000: Client c504f2f9-b6ab-4ec1-b8de-dc9733b86500 (at 192.168.203.17@tcp) reconnecting [ 3182.564030] Lustre: *** cfs_fail_loc=159, val=0*** [ 3182.565782] Lustre: Skipped 14 previous similar messages [ 3198.764961] Lustre: lustre-MDT0000: Client c504f2f9-b6ab-4ec1-b8de-dc9733b86500 (at 192.168.203.17@tcp) reconnecting [ 3199.372352] Lustre: 139285:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3203.044719] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3203.044780] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3203.045441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3203.049922] Lustre: Skipped 1 previous similar message [ 3207.011432] Lustre: server umount lustre-OST0000 complete [ 3209.183875] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3209.187282] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3209.192303] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3209.196174] Lustre: Skipped 1 previous similar message [ 3213.280074] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3213.280595] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3214.760043] Lustre: server umount lustre-MDT0000 complete [ 3216.133205] LustreError: 135944:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784406210 with bad export cookie 17067343877495015253 [ 3216.136584] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3216.138112] LustreError: 135944:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3216.259018] Lustre: server umount lustre-MDT0001 complete [ 3220.234466] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 3221.460539] Key type lgssc unregistered [ 3221.595545] LNet: 140334:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3221.599249] LNetError: 140334:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3221.606389] LNet: Removed LNI 192.168.203.117@tcp [ 3221.977191] Key type .llcrypt unregistered [ 3221.979138] Key type ._llcrypt unregistered [ 3232.400525] Key type ._llcrypt registered [ 3232.401988] Key type .llcrypt registered [ 3232.449810] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3232.776474] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3232.789841] alg: No test for adler32 (adler32-zlib) [ 3233.687413] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 3233.808025] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 3235.407244] Key type lgssc registered [ 3235.925323] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3238.909387] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3238.915871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3240.077747] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3241.655841] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3244.734539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3244.863887] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3246.277451] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3248.947955] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3251.064525] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3253.616993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3253.734664] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3254.759382] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3255.799335] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3258.393567] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3267.051774] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3272.315661] Lustre: 144328:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3273.100099] Lustre: 144377:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3273.712673] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3275.107131] Lustre: 144523:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3275.463742] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3275.467322] Lustre: Skipped 1 previous similar message [ 3275.812512] Lustre: 144621:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3277.281673] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3277.282033] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3277.282943] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3277.287259] Lustre: Skipped 1 previous similar message [ 3278.998930] Lustre: server umount lustre-OST0000 complete [ 3280.864086] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3280.867370] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3280.873285] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3280.876031] Lustre: Skipped 1 previous similar message [ 3282.400183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3282.401077] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3282.472811] Lustre: server umount lustre-MDT0000 complete [ 3283.901447] LustreError: 143566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784406277 with bad export cookie 6733646358406351669 [ 3283.904429] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3283.909208] LustreError: 143566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3284.038894] Lustre: server umount lustre-MDT0001 complete [ 3288.068930] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 3289.250475] Key type lgssc unregistered [ 3289.388452] LNet: 145677:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3289.393099] LNetError: 145677:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3289.402287] LNet: Removed LNI 192.168.203.117@tcp [ 3289.761591] Key type .llcrypt unregistered [ 3289.763168] Key type ._llcrypt unregistered [ 3299.853358] Key type ._llcrypt registered [ 3299.854960] Key type .llcrypt registered [ 3299.902325] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3300.265870] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3300.315268] alg: No test for adler32 (adler32-zlib) [ 3301.190571] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 3301.291643] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 3302.887154] Key type lgssc registered [ 3303.336217] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3306.299806] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3306.308244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3307.475784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3309.016651] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3312.399678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3312.572403] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3314.079699] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3316.804188] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3318.781888] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3321.352600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3321.505118] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3322.535164] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3323.810267] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3326.558881] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3332.775962] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3339.324553] Lustre: *** cfs_fail_loc=159, val=0*** [ 3339.326908] Lustre: Skipped 2 previous similar messages [ 3355.945032] Lustre: lustre-MDT0000: Client 41c848a4-2cbc-4c13-a918-5af76ea2e73f (at 192.168.203.17@tcp) reconnecting [ 3374.051379] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3374.051574] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3374.057105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3374.059528] LustreError: Skipped 1 previous similar message [ 3374.062978] Lustre: Skipped 1 previous similar message [ 3377.046956] Lustre: server umount lustre-OST0000 complete [ 3379.169600] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3379.169644] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3379.174357] Lustre: Skipped 1 previous similar message [ 3379.178493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3379.182769] Lustre: Skipped 1 previous similar message [ 3384.288253] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3384.291575] Lustre: Skipped 1 previous similar message [ 3384.603718] Lustre: server umount lustre-MDT0000 complete [ 3386.190326] LustreError: 149701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784406380 with bad export cookie 1866840574942925052 [ 3386.193582] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3386.194708] LustreError: 149701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3386.325888] Lustre: server umount lustre-MDT0001 complete [ 3390.597129] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 3391.791532] Key type lgssc unregistered [ 3391.946152] LNet: 150754:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3391.951710] LNetError: 150754:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3391.965555] LNet: Removed LNI 192.168.203.117@tcp [ 3392.350446] Key type .llcrypt unregistered [ 3392.352114] Key type ._llcrypt unregistered [ 3402.660179] Key type ._llcrypt registered [ 3402.661885] Key type .llcrypt registered [ 3402.708251] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3403.197111] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3403.218932] alg: No test for adler32 (adler32-zlib) [ 3404.095984] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 3404.196085] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 3405.791385] Key type lgssc registered [ 3406.199170] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3409.154363] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3409.164648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3410.344068] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3411.854221] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3415.009813] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3415.167274] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3416.817114] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3419.739960] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3421.966538] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3424.517697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3424.661261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3425.703066] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3426.835107] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3429.539598] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3435.942291] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3453.122127] Lustre: lustre-OST0000: evicting 16a67139-dee4-4aca-897d-b2e0b02b68be (at 192.168.203.17@tcp) by administrative request [ 3453.127682] Lustre: Skipped 1 previous similar message [ 3482.084911] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3482.086616] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3482.087112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3482.090451] Lustre: Skipped 1 previous similar message [ 3482.093615] LustreError: Skipped 1 previous similar message [ 3483.680511] Lustre: server umount lustre-OST0000 complete [ 3486.688093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3486.691992] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3486.699761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3486.703202] Lustre: Skipped 1 previous similar message [ 3491.383699] Lustre: server umount lustre-MDT0000 complete [ 3492.321761] LustreError: 152890:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3492.331545] LustreError: 152890:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3493.055610] LustreError: 153995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784406487 with bad export cookie 5505838548500317370 [ 3493.059195] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3493.060683] LustreError: 153995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3493.221254] Lustre: server umount lustre-MDT0001 complete [ 3497.628031] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 3498.871669] Key type lgssc unregistered [ 3499.049673] LNet: 156234:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3499.054447] LNetError: 156234:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3499.069780] LNet: Removed LNI 192.168.203.117@tcp [ 3499.477875] Key type .llcrypt unregistered [ 3499.479667] Key type ._llcrypt unregistered [ 3529.856623] Key type ._llcrypt registered [ 3529.858010] Key type .llcrypt registered [ 3529.907135] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3530.262505] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3530.274765] alg: No test for adler32 (adler32-zlib) [ 3531.161664] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 3531.268821] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 3532.879161] Key type lgssc registered [ 3533.331479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3536.686922] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3536.697117] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3537.919333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3539.674267] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3543.545927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3543.733620] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3545.441644] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3548.559535] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3550.980396] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3554.147265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3554.367405] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3557.017592] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3560.311717] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3562.604065] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3568.742119] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3578.336043] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3578.337618] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3578.338758] LustreError: Skipped 1 previous similar message [ 3578.347454] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3579.361129] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3579.361181] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3579.364565] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3580.697131] Lustre: server umount lustre-OST0000 complete [ 3584.480617] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3584.481850] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3584.488391] Lustre: Skipped 1 previous similar message [ 3584.493205] Lustre: Skipped 2 previous similar messages [ 3588.380884] Lustre: server umount lustre-MDT0000 complete [ 3589.600135] LustreError: 158689:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3589.606279] LustreError: 158689:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3589.845256] LustreError: 158676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784406583 with bad export cookie 2992588773800912450 [ 3589.848391] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3589.853080] LustreError: 158676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3589.973680] Lustre: server umount lustre-MDT0001 complete [ 3594.125941] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 3595.316726] Key type lgssc unregistered [ 3595.470555] LNet: 162251:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3595.476108] LNetError: 162251:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3595.486335] LNet: Removed LNI 192.168.203.117@tcp [ 3595.858366] Key type .llcrypt unregistered [ 3595.859792] Key type ._llcrypt unregistered [ 3606.118686] Key type ._llcrypt registered [ 3606.120403] Key type .llcrypt registered [ 3606.161738] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3606.579190] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3606.588520] alg: No test for adler32 (adler32-zlib) [ 3607.448132] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 3607.544587] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 3609.129835] Key type lgssc registered [ 3609.572048] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3615.714238] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 16:30:07 (1784406607) [ 3622.299426] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 3626.276144] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3640.780854] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3640.791379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3641.904117] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3641.922355] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3641.969513] Lustre: lustre-MDT0000: new disk, initializing [ 3642.003652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3642.012491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3643.876940] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3648.974543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3649.027462] Lustre: 166482:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3649.055488] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3649.057772] Lustre: Skipped 1 previous similar message [ 3649.096247] Lustre: lustre-MDT0001: new disk, initializing [ 3649.138948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3649.156136] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3649.160964] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3650.958472] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3655.330715] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3657.720300] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3660.493553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3660.611504] Lustre: lustre-OST0000: new disk, initializing [ 3660.613981] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3660.618285] Lustre: 168450:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3660.660677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3663.016566] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3667.289840] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3668.993098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3668.998828] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3669.043958] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3679.205768] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3679.205849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3679.213253] LustreError: Skipped 1 previous similar message [ 3679.221112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3679.226992] Lustre: Skipped 1 previous similar message [ 3683.218267] Lustre: server umount lustre-OST0000 complete [ 3685.343898] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3685.346992] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3685.352588] Lustre: Skipped 1 previous similar message [ 3685.354615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3689.440610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3689.442039] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3690.720106] Lustre: server umount lustre-MDT0000 complete [ 3692.240611] LustreError: 166476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784406686 with bad export cookie 2010825474278910138 [ 3692.243390] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3692.246492] LustreError: 166476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3692.390878] Lustre: server umount lustre-MDT0001 complete [ 3696.724832] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 3697.904913] Key type lgssc unregistered [ 3698.039492] LNet: 170353:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3698.044309] LNetError: 170353:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3698.058944] LNet: Removed LNI 192.168.203.117@tcp [ 3698.411267] Key type .llcrypt unregistered [ 3698.412839] Key type ._llcrypt unregistered [ 3708.548933] Key type ._llcrypt registered [ 3708.550492] Key type .llcrypt registered [ 3708.596959] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 3709.072960] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3709.103263] alg: No test for adler32 (adler32-zlib) [ 3709.979075] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 3710.084572] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 3711.679228] Key type lgssc registered [ 3712.128211] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3715.301480] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3715.312253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3716.523835] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3718.294804] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3721.720596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3721.894155] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3723.553109] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3726.449718] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3728.593279] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3731.270846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3731.401597] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3733.583643] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 3736.338475] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3741.671760] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3750.296517] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3750.307527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3750.313621] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3750.318698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.324824] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3750.747952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.124486] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3751.128974] Lustre: Skipped 1 previous similar message [ 3751.136509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3751.143670] Lustre: Skipped 1 previous similar message [ 3751.146474] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3751.150632] Lustre: Skipped 1 previous similar message [ 3751.152856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.157985] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3751.160671] Lustre: Skipped 1 previous similar message [ 3751.507737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.867520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.215244] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3752.219919] Lustre: Skipped 2 previous similar messages [ 3752.227517] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3752.233467] Lustre: Skipped 2 previous similar messages [ 3752.235905] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3752.238938] Lustre: Skipped 2 previous similar messages [ 3752.242058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.248234] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3752.251429] Lustre: Skipped 2 previous similar messages [ 3752.624484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.037956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.450510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.797525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.165477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.485322] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3754.489655] Lustre: Skipped 5 previous similar messages [ 3754.497211] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3754.503779] Lustre: Skipped 5 previous similar messages [ 3754.506393] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3754.509152] Lustre: Skipped 5 previous similar messages [ 3754.511159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.515392] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3754.519725] Lustre: Skipped 5 previous similar messages [ 3754.855155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.197817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.536579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.885055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.219101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.555566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.877312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.222410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.557277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.868238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.176563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.515429] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3758.518668] Lustre: Skipped 11 previous similar messages [ 3758.526112] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3758.531909] Lustre: Skipped 11 previous similar messages [ 3758.533643] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3758.536404] Lustre: Skipped 11 previous similar messages [ 3758.538333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.543273] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3758.545801] Lustre: Skipped 11 previous similar messages [ 3758.878482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.199487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.527532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.871497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.193259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.514071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.840682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.170957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.473076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.775949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.101817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.449022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.763265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.079943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.402534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.727000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.048186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.364743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.680566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.015360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.333042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.626961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.908534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.212789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.519684] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3766.522442] Lustre: Skipped 24 previous similar messages [ 3766.527324] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3766.531793] Lustre: Skipped 24 previous similar messages [ 3766.533888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.833964] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3766.836457] Lustre: Skipped 25 previous similar messages [ 3766.837721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.842871] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3766.845236] Lustre: Skipped 25 previous similar messages [ 3767.165796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.488283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.803409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.121020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.443230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.810485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.141750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.474482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.806665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.154499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.493398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.819216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.152222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.473551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.788964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.093133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.397257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.716575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.042956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.354370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.677729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.994524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.294016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.584616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.905687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.248244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.577294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.876342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.194242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.507472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.835881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.139497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.442966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.774843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.104304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.424310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.751541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.102519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.435336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.750087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.076891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.403134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.705719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.018302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.338321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.664640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.982917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.320525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.649026] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3782.652654] Lustre: Skipped 49 previous similar messages [ 3782.658433] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3782.662882] Lustre: Skipped 49 previous similar messages [ 3782.664635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.001351] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3783.003583] Lustre: Skipped 49 previous similar messages [ 3783.005362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.010963] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3783.013132] Lustre: Skipped 49 previous similar messages [ 3783.344104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.669218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.992488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.338599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.659202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.997938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3785.315862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3785.648209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3785.965531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3786.305485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3786.660125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.012307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.331695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.670839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3788.027605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3788.381382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3788.731223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.067683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.406256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.732031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.084775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.413769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.769867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.086045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.430650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.756612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.108237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.437268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.752759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.066790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.410993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.722127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3794.026358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3794.339969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3794.684217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3794.999538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3795.318454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3795.668271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.041389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.364783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.736064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3797.098415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3797.445132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3797.786698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.138664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.505340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.843607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.175164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.527655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.879297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.228588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.604865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.953283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3801.302471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3801.655693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.030980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.447723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.820396] LustreError: 171762:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3802.825964] LustreError: 171762:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3802.826847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.204833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.546622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.913287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3804.257417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3804.613957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3804.970209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.328610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.703865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.061179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.405601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.773756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.178906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.581943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.960079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.338757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.690219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3809.087755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3809.514626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3809.892349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.237610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.601583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.974024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3811.346246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3811.744476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.120939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.544968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.983995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3813.379802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3813.746853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.143831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.504360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.865790] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3814.868974] Lustre: Skipped 90 previous similar messages [ 3814.875095] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3814.879515] Lustre: Skipped 90 previous similar messages [ 3814.881599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.241327] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3815.245728] Lustre: Skipped 90 previous similar messages [ 3815.248126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.254619] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3815.257476] Lustre: Skipped 90 previous similar messages [ 3815.608418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.965703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3816.314989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3816.645879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3816.996859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.333722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.688179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.016049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.354703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.690858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.044254] LustreError: 171762:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3819.049218] LustreError: 171762:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3819.049541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.411323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.731503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.052840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.380797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.702294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.041046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.387679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.689145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.016726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.352729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.686007] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.021377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.356937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.709282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.035563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.349546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.668377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.992521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.318093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.640056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.985557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3826.362358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3826.696569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.010291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.345068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.696783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.057389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.395623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.753377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3829.125316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3829.485950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3829.857340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.223443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.582439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.972391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.353628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.712596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.038414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.391184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.751128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.073680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.408373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.719041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.044663] LustreError: 171762:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3834.049185] LustreError: 171762:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3834.049769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.371902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.685468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.002065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.347353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.658392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.986961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.315464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.639178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.971724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3837.295851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3837.640558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.013505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.341746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.670852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.000535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.325418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.633559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.947567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.253126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.560193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.855187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.164536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.475743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.786844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.095067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.410547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.714724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.042788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.351888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.663754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.978652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.318628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.637098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.956134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.307357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.668202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.004987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.347867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.678568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.019543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.353105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.676992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.003261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.337278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.669635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.019649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.361631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.685879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.037538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.383304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.736280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.081678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.439276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.808412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.196334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.565778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.944750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.302677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.668455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.007785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.333780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.676869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.009387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.338417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.691462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.039833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.390382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.742123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.111346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.482518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.830244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.156620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.485794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.809153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.164734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.460487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.786962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.112994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.452714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.773259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.102500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.423394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.768458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.108777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.452831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.790848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.165256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.522719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.876469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.221571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.581106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.908602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.253833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.591258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.938385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.250431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.567676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.889044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.250345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.598731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.933255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.264236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.614549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.948768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.287404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.626337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.966672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.279347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.602554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.933345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.291308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.628761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.966542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.307834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.692067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.034877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.359126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.691498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.030851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.370468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.710966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.063502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.456309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.841071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.239779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.595983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.937052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.284615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.638423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.024937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.373375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.713701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.022143] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3879.024666] Lustre: Skipped 188 previous similar messages [ 3879.031376] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3879.035685] Lustre: Skipped 188 previous similar messages [ 3879.037498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.353035] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3879.355890] Lustre: Skipped 188 previous similar messages [ 3879.357725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.363401] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3879.365205] Lustre: Skipped 188 previous similar messages [ 3879.685752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.995951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.317976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.649752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.976863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.313248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.636358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.978106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.286273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.620207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.939718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.269133] LustreError: 171762:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3883.272470] LustreError: 171762:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3883.274223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.608527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.932636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.277756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.648224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.010124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.393401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.764484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.107886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.448347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.759207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.071670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.371406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.680777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.983434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.302930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.629435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.950822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.268231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.616224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.931577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.249772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.565580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.898566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.220296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.539649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.845226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.171333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.483453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.826459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.173258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.533866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.886524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.225089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.571904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.898127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.233969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.567324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.919713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.266294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.601927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.946206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.295140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.646942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.970188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.320975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.680662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.041751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.438598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.847237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.271700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.661664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.043959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.443250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.818707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.193937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.595693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.995223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.385906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.782458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.154268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.587960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.954859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.333331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.693582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.093803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.486696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.850460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.235499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.639838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.023769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.402167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.808951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.185230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.570043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.962810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.390965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.794764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.167715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.557875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.938723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.294674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.651446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.018371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.392961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.764357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.136325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.488879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.859646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.240861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.659033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.036244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.429867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.825220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.176140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.541799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.915620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.307882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.671483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.033793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.368569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.703236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.026108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.367425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.695779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.070147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.420291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.788262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.150958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.540500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.915577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.288418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.632437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.001363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.379943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.736282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.128150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.496849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.889349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.318454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.710690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.088329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.500346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.897694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.298870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.681844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.106285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.519115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.937901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.357588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.741482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.125799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.500313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.883336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.251340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.612542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.998037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.412768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.763644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.124756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.519824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.893185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.277073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.635437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.982869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.344121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.727172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.102788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.463050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.841800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.221092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.589041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.940992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.310596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.663352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.016835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.381928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.743971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.111551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.507755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.877262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.255568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.647255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.029908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.403930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.807227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.186216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.542284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.901734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.264669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.595429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.935949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.269836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.612819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.929198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.271892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.621203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.985560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.337976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.725418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.098208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.431967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.752536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.075217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.402057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.707859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.011312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.343365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.674536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.018859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.353137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.681125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.044958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.364441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.677687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.999439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.355176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.673159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.986082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.329899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.661977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.980167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.316030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.650196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.991274] LustreError: 171762:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3956.994474] LustreError: 171762:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3956.996775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.336822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.664617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.002641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.344021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.663360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.971613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.293654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.639793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.966731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.290807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.613241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.944801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.261289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.578142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.902731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.229883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.553312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.871995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.193295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.542048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.859561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.168236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.469297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.774534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.076354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.373920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.680825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.033510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.338189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.654893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.963637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.299285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.619270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.979591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.317435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.645198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.964343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.296363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.609610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.943562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.255868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.550589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.864735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.187361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.509295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.842173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.178673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.520942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.838681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.160812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.475444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.822123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.156275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.475860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.827319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.192839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.542054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.873218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.233619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.576843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.910950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.235184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.588448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.907645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.259970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.580876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.913490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.224526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.538067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.845850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.186354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.502732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.806922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.124348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.476536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.798693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.111627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.434542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.784048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.114651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.434968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.763071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.118687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.451045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.806837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.134015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.473918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.846267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.186929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.520331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.850139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.183375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.523567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.856638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.191307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.516503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.865576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.188380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.521827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.852447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.184464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.497922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.826848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.172359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.526704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.861346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.197582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.521438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.884169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.236339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.575044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.921117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.291077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.635320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.975375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.304437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.659675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.991173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.339559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.685189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.020179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.333076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.649101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.975208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.350423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.671798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.017371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.356222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.712394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.057446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.402794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.739335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.078277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.442432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.820160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.165989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.510493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.851317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.221717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.572969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.922953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.256309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.609403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.976301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.342900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.717723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.122552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.491288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.873193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.215521] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4007.219061] Lustre: Skipped 366 previous similar messages [ 4007.226787] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4007.232711] Lustre: Skipped 366 previous similar messages [ 4007.234540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.578141] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4007.581155] Lustre: Skipped 366 previous similar messages [ 4007.583284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.589408] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4007.592209] Lustre: Skipped 366 previous similar messages [ 4007.934804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.299282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.641437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.010676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.339912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.681517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.005724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.318428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.656470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.990792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.315475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.657696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.018704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.344843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.665067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.010425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.385701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.703427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.042720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.386668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.746029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.078596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.408970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.750866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.095830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.427431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.770230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.110027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.432236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.746201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.065270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.397125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.746924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.110795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.470268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.814403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.160537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.489813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.851878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.176711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.496848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.817204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.154898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.492942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.834239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.164279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.518376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.866677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.215427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.573332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.934434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.265883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.600158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.982586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.322502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.656986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.981311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.332498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.652210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.994322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.339592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.674098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.992575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.354787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.685439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.029869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.348361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.669480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.998980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.358730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.683655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.014328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.344446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.695775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.027254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.367408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.713296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.053204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.380203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.700784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.029601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.384552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.707695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.040307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.371265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.735448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.082273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.420312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.756930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.103690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.424894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.740521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.078528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.431258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.758078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.080328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.425755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.779651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.114340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.442171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.775204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.131988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.471695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.812070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.135421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.499233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.839793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.163394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.474040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.807513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.126567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.431789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.750107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.085744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.416560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.739286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.049611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.376916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.700242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.040741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.369284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.706635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.056377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.400633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.729474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.078297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.408198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.742848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.063450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.394427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.704083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.004065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.319133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.642989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.932490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.219640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.511644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.817617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.132844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.459218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.789617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.159344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.501282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.860101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.212604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.574392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.906480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.234206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.553817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.889346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.207670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.525320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.835231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.181829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.537246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.880741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.222582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.558124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.897457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.233618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.579826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.941044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.318870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.683459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.025888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.366121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.715711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.052472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.422314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.748049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.078609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.405054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.732771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.118178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.456177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.795138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.135407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.553364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.920111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.249591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.586056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.910419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.234738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.578709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.945209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.301915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.632247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.977414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.336755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.652848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.957782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.252240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.558352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.864847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.160168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.468136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.794197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.088382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.394081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.717456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.068397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.382711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.710637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.037926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.357787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.660674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.966397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.286605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.605242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.913132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.218872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.533972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.862795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.200426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.536885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.859126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.186994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.502906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.827972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.133063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.480280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.790434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.129232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.485769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.857442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.212906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.557461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.925151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.244452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.592563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.931571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.285973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.606093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.949166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.289169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.662748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.989166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.323389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.698539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.025615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.377479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.725244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.060379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.414978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.754402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.103923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.451833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.820449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.144882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.541906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.907770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.276114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.655418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.547066] LustreError: 174241:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4218.551710] LustreError: 171762:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 4335.079638] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4335.079706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4335.083566] LustreError: Skipped 1 previous similar message [ 4335.087893] Lustre: Skipped 254 previous similar messages [ 4335.088649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4335.586084] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4335.589519] LustreError: Skipped 3 previous similar messages [ 4340.708512] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4340.711914] Lustre: Skipped 2 previous similar messages [ 4345.826123] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4345.830067] Lustre: Skipped 1 previous similar message [ 4346.335350] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4346.454804] Lustre: server umount lustre-OST0000 complete [ 4350.946401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4350.949859] Lustre: Skipped 1 previous similar message [ 4356.064096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4356.067413] Lustre: Skipped 1 previous similar message [ 4362.207194] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4362.324614] Lustre: server umount lustre-MDT0000 complete [ 4363.606974] LustreError: 171723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407357 with bad export cookie 12013663122186741685 [ 4363.610738] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4363.612437] LustreError: 171723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4363.742963] Lustre: server umount lustre-MDT0001 complete [ 4367.451919] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 4368.541243] Key type lgssc unregistered [ 4368.678332] LNet: 226792:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4368.683152] LNetError: 226792:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4368.691354] LNet: Removed LNI 192.168.203.117@tcp [ 4369.014444] Key type .llcrypt unregistered [ 4369.015408] Key type ._llcrypt unregistered [ 4378.366718] Key type ._llcrypt registered [ 4378.368331] Key type .llcrypt registered [ 4378.409069] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 4378.763988] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4378.772392] alg: No test for adler32 (adler32-zlib) [ 4379.646903] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 4379.751099] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 4381.343161] Key type lgssc registered [ 4381.775335] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4384.940963] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4384.947736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4386.114793] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4387.630166] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4390.711053] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4390.838744] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4392.198705] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4394.839183] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4396.891403] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4399.368362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4399.550101] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4401.814502] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4404.416386] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4411.878561] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 4416.728741] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4417.353673] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4417.357429] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4417.361390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.365448] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4417.365491] LustreError: 228205:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 4419.873581] Lustre: lustre-OST0000: Not available for connect from 192.168.203.17@tcp (stopping) [ 4420.002062] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4424.984841] Lustre: lustre-OST0000: Not available for connect from 192.168.203.17@tcp (stopping) [ 4424.987425] Lustre: Skipped 2 previous similar messages [ 4430.105054] Lustre: lustre-OST0000: Not available for connect from 192.168.203.17@tcp (stopping) [ 4430.108214] Lustre: Skipped 2 previous similar messages [ 4432.351155] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4432.416065] Lustre: server umount lustre-OST0000 complete [ 4437.447141] LustreError: 228205:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 4460.563539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4460.709298] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4461.736083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.745642] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.745714] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4461.756183] Lustre: Skipped 1 previous similar message [ 4462.955401] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4465.663729] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4466.304564] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 4468.420333] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff996bc55d6800.ost_server_uuid 50 [ 4469.031071] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff996bc55d6800.ost_server_uuid in FULL state after 0 sec [ 4472.289647] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4472.289646] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4472.290568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4472.293578] Lustre: Skipped 2 previous similar messages [ 4472.296091] Lustre: Skipped 2 previous similar messages [ 4476.170812] Lustre: server umount lustre-OST0000 complete [ 4477.408726] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4477.409456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4477.411536] Lustre: Skipped 1 previous similar message [ 4477.412723] Lustre: Skipped 1 previous similar message [ 4483.539683] Lustre: server umount lustre-MDT0000 complete [ 4484.782187] LustreError: 228167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407478 with bad export cookie 14675253231670944204 [ 4484.785320] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4484.788309] LustreError: 228167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4484.903326] Lustre: server umount lustre-MDT0001 complete [ 4488.535664] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 4489.574455] Key type lgssc unregistered [ 4489.697375] LNet: 233411:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4489.700131] LNetError: 233411:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4489.706301] LNet: Removed LNI 192.168.203.117@tcp [ 4490.030766] Key type .llcrypt unregistered [ 4490.032420] Key type ._llcrypt unregistered [ 4500.895824] Key type ._llcrypt registered [ 4500.896643] Key type .llcrypt registered [ 4500.928983] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 4501.256214] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4501.309636] alg: No test for adler32 (adler32-zlib) [ 4502.168714] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 4502.259553] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 4503.839206] Key type lgssc registered [ 4504.178897] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4507.241430] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4507.247128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4508.400486] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4509.696574] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4512.538479] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4512.669476] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4513.996188] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4514.975411] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4517.085447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4517.203787] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4518.249534] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 4519.286264] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4521.953796] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4522.014712] Lustre: lustre-OST0001: new disk, initializing [ 4522.016805] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4522.019365] Lustre: 237423:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4522.056167] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4522.941328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4522.945429] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4522.978914] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4523.905902] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4529.102809] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4533.670762] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4543.456170] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4543.459703] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4543.466041] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4544.484240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4544.484726] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4544.492918] Lustre: Skipped 2 previous similar messages [ 4547.102287] Lustre: server umount lustre-MDT0000 complete [ 4548.235390] LustreError: 235097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407542 with bad export cookie 11996961864162594924 [ 4548.237921] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4548.241418] LustreError: 235097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4548.337735] Lustre: server umount lustre-MDT0001 complete [ 4559.816653] Lustre: server umount lustre-OST0000 complete [ 4570.275468] Lustre: server umount lustre-OST0001 complete [ 4577.139122] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 4580.606052] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4580.760786] LustreError: 240532:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4580.768727] LustreError: 240532:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4580.785140] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4582.130921] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4585.208615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4586.776557] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4587.801365] Lustre: 241674:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4590.097868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4590.224743] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4590.227933] Lustre: Skipped 1 previous similar message [ 4591.267718] LustreError: 242026:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4591.271585] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 4592.201577] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4595.180123] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4597.361739] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4600.644882] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4601.902636] Lustre: 243540:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4606.433351] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4606.436418] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4606.438593] Lustre: Skipped 2 previous similar messages [ 4606.440463] Lustre: Skipped 4 previous similar messages [ 4611.552813] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4611.555821] Lustre: Skipped 1 previous similar message [ 4612.452952] Lustre: server umount mylustre-MDT0000 complete [ 4613.735164] LustreError: 241277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407607 with bad export cookie 11996961864162597689 [ 4613.738438] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4613.739820] LustreError: 241277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4613.844654] Lustre: server umount mylustre-MDT0001 complete [ 4625.921222] Lustre: server umount mylustre-OST0000 complete [ 4637.466497] Lustre: server umount mylustre-OST0001 complete [ 4644.462372] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 4648.133490] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4648.283929] LustreError: 245850:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4648.304538] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4648.306896] Lustre: Skipped 1 previous similar message [ 4649.793132] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4652.790897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4654.316195] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4655.332437] Lustre: 246992:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4657.652335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4658.787771] LustreError: 247347:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4658.793840] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 4659.839526] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4662.885829] LustreError: 247528:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4662.892245] LustreError: 247528:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4662.976145] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4665.131382] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4668.401847] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4669.695054] Lustre: 248861:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4673.505809] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4673.506149] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4673.510276] Lustre: Skipped 3 previous similar messages [ 4673.510687] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4677.726216] Lustre: server umount tfs-MDT0000 complete [ 4678.625393] LustreError: 245850:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4678.631646] LustreError: 245850:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4679.074298] LustreError: 247371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407673 with bad export cookie 11996961864162599355 [ 4679.076921] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4679.080177] LustreError: 247371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4679.205590] Lustre: server umount tfs-MDT0001 complete [ 4690.952202] Lustre: server umount tfs-OST0000 complete [ 4702.554303] Lustre: server umount tfs-OST0001 complete [ 4709.374253] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 4712.836726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4712.985037] LustreError: 251109:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4713.002657] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4713.004650] Lustre: Skipped 3 previous similar messages [ 4714.344107] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4717.206138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4718.728387] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4719.688690] Lustre: 252250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4721.828368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4722.984520] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 4723.867203] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4726.825293] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4728.910926] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4732.039481] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4733.184598] Lustre: 254119:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4735.354794] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 16:48:47 (1784407727) [ 4748.256042] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4748.260623] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4748.267282] Lustre: Skipped 1 previous similar message [ 4748.269826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4748.271740] Lustre: Skipped 3 previous similar messages [ 4753.387062] Lustre: server umount lustre-MDT0000 complete [ 4753.889462] LustreError: 253584:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4753.896053] LustreError: 253584:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4754.708182] LustreError: 251086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407748 with bad export cookie 11996961864162600790 [ 4754.713066] LustreError: 251086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4754.713267] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4759.009035] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4759.014637] Lustre: Skipped 4 previous similar messages [ 4769.247130] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4769.248278] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4769.252587] Lustre: Skipped 8 previous similar messages [ 4769.339891] Lustre: server umount lustre-MDT0001 complete [ 4776.767146] Lustre: server umount lustre-OST0000 complete [ 4792.287197] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4792.361895] Lustre: server umount lustre-OST0001 complete [ 4794.654946] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 4797.944471] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 4810.783699] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4810.783699] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4810.783699] [ 4810.791668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4810.870110] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4810.882297] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4810.920317] Lustre: lustre-MDT0000: new disk, initializing [ 4810.944538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4810.947015] Lustre: Skipped 3 previous similar messages [ 4810.954256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4812.381986] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4816.654559] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 4816.654559] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4816.654559] [ 4816.662473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4816.704422] Lustre: 257296:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4816.731414] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4816.733908] Lustre: Skipped 1 previous similar message [ 4816.774624] Lustre: lustre-MDT0001: new disk, initializing [ 4816.826423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4816.830098] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4818.485360] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4822.793338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4822.922285] Lustre: lustre-OST0000: new disk, initializing [ 4822.924695] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4822.928520] Lustre: 258879:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4824.954214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4824.958119] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4824.959392] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4824.992160] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4829.318163] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4829.385332] Lustre: lustre-OST0001: new disk, initializing [ 4829.387759] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4829.391290] Lustre: 259750:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4830.531034] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4830.534510] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4830.567734] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4831.551145] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4836.097904] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 16:50:28 (1784407828) [ 4839.882935] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 16:50:32 (1784407832) [ 4855.779069] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4855.779302] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4855.782623] LustreError: Skipped 1 previous similar message [ 4855.792342] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4855.796817] Lustre: Skipped 1 previous similar message [ 4857.620892] Lustre: server umount lustre-OST0000 complete [ 4861.414952] LustreError: 258864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4861.434901] LustreError: 258864:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 4867.019627] Lustre: server umount lustre-MDT0000 complete [ 4869.678919] LustreError: 257288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407863 with bad export cookie 11996961864162602120 [ 4869.687236] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4869.687404] LustreError: 257288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4876.147901] Lustre: server umount lustre-MDT0001 complete [ 4885.484261] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 4902.664776] Lustre: server umount lustre-OST0001 complete [ 4906.773297] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 4915.651972] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 4935.580866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4935.725990] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4935.743200] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4935.791415] Lustre: lustre-MDT0000: new disk, initializing [ 4935.842886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4937.914323] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4943.382783] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4943.440528] Lustre: 264280:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4943.563623] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4943.568130] Lustre: Skipped 4 previous similar messages [ 4943.584539] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4945.426831] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4950.818110] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4953.649065] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4957.274561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4957.432588] Lustre: lustre-OST0000: new disk, initializing [ 4957.435194] Lustre: Skipped 1 previous similar message [ 4957.438892] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4957.442120] Lustre: Skipped 2 previous similar messages [ 4957.446237] Lustre: 266245:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4959.363940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4959.369076] Lustre: Skipped 1 previous similar message [ 4959.374850] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4959.426330] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4960.879151] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 4966.167468] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4969.953123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4969.957083] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4969.962851] Lustre: Skipped 5 previous similar messages [ 4969.967099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4969.972638] Lustre: Skipped 11 previous similar messages [ 4975.534666] Lustre: server umount lustre-OST0000 complete [ 4979.681775] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4979.691269] LustreError: Skipped 1 previous similar message [ 4983.816889] Lustre: server umount lustre-MDT0000 complete [ 4985.763451] LustreError: 264269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784407979 with bad export cookie 11996961864162603590 [ 4985.767589] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4985.770623] LustreError: 264269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4985.942304] Lustre: server umount lustre-MDT0001 complete [ 4990.947262] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 4992.252881] Key type lgssc unregistered [ 4992.422113] LNet: 268002:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4992.428574] LNetError: 268002:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4992.439750] LNet: Removed LNI 192.168.203.117@tcp [ 4992.879262] Key type .llcrypt unregistered [ 4992.881533] Key type ._llcrypt unregistered [ 5006.982805] Key type ._llcrypt registered [ 5006.987116] Key type .llcrypt registered [ 5007.053181] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5007.421991] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5007.447790] alg: No test for adler32 (adler32-zlib) [ 5008.347443] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 5008.472484] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 5010.079206] Key type lgssc registered [ 5010.645425] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5014.305099] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5014.315343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5015.570544] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5017.738269] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5022.113027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5022.338190] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5024.898457] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5029.524471] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5032.844172] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5036.977945] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5037.183017] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5040.236765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5040.507533] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5044.292646] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5044.841193] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5047.980668] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5047.981605] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5047.995447] Lustre: Skipped 1 previous similar message [ 5052.064301] Lustre: server umount lustre-OST0000 complete [ 5058.016452] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5058.021953] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5058.033581] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5063.648846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5063.652849] Lustre: Skipped 2 previous similar messages [ 5068.255126] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5068.349254] Lustre: server umount lustre-MDT0000 complete [ 5068.768171] LustreError: 270172:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5068.779828] LustreError: 270172:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5070.521178] LustreError: 269424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408064 with bad export cookie 14829128366277004537 [ 5070.524798] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5070.527491] LustreError: 269424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5070.730723] Lustre: server umount lustre-MDT0001 complete [ 5076.425616] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 5077.971537] Key type lgssc unregistered [ 5078.152884] LNet: 273032:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5078.156728] LNetError: 273032:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5078.163527] LNet: Removed LNI 192.168.203.117@tcp [ 5078.616600] Key type .llcrypt unregistered [ 5078.617827] Key type ._llcrypt unregistered [ 5086.355326] Key type ._llcrypt registered [ 5086.356431] Key type .llcrypt registered [ 5086.399670] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5086.752260] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5086.787032] alg: No test for adler32 (adler32-zlib) [ 5087.654545] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 5087.763348] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 5089.359153] Key type lgssc registered [ 5089.860719] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5093.371698] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5093.377725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5108.831385] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5113.951298] LustreError: 274203:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.117@tcp: failed processing log, type 4: rc = -110 [ 5140.575158] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5142.918117] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5145.908922] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5146.785407] Lustre: server umount lustre-OST0000 complete [ 5153.250643] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 5154.436994] Key type lgssc unregistered [ 5154.593428] LNet: 275830:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5154.596684] LNetError: 275830:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5154.609490] LNet: Removed LNI 192.168.203.117@tcp [ 5154.932200] Key type .llcrypt unregistered [ 5154.933672] Key type ._llcrypt unregistered [ 5170.917012] Key type ._llcrypt registered [ 5170.918868] Key type .llcrypt registered [ 5170.966980] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5171.435813] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5171.447125] alg: No test for adler32 (adler32-zlib) [ 5172.313808] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 5172.414364] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 5173.999187] Key type lgssc registered [ 5174.391473] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5179.043469] loop0: detected capacity change from 0 to 204800000 [ 5179.121562] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5179.129233] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5179.284235] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5179.673747] loop0: detected capacity change from 0 to 204800000 [ 5179.737965] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5179.743463] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5179.889685] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5180.306571] loop0: detected capacity change from 0 to 204800000 [ 5180.372539] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5180.376757] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5180.425518] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5180.862186] loop0: detected capacity change from 0 to 204800000 [ 5180.941075] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5181.451228] loop0: detected capacity change from 0 to 204800000 [ 5181.461744] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5183.049284] loop0: detected capacity change from 0 to 204800000 [ 5183.060486] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5184.605399] loop0: detected capacity change from 0 to 204800000 [ 5184.617392] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5186.153529] loop0: detected capacity change from 0 to 204800000 [ 5186.167943] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5187.765801] loop0: detected capacity change from 0 to 204800000 [ 5187.800491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5187.811735] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5187.865988] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5187.872359] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 5187.877368] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5187.883267] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 5187.888927] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 5187.894865] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5187.899738] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5187.904966] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 5187.910563] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 5187.916583] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 5187.921810] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 5187.927036] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 5187.932513] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 5187.937646] Lustre: 279222:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 5190.794799] Lustre: server umount MGS complete [ 5191.125059] loop0: detected capacity change from 0 to 204800000 [ 5191.154041] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5191.348888] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5191.369226] Lustre: lustre-MDT0000: Aborting client recovery [ 5191.718554] loop1: detected capacity change from 0 to 204800000 [ 5191.750456] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5191.807942] Lustre: 279655:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5191.814317] Lustre: 279655:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5191.817898] Lustre: 279655:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 5191.821572] Lustre: 279655:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5191.827500] Lustre: 279655:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5191.832621] Lustre: 279655:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 5191.836796] Lustre: 279655:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 5191.846850] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5191.927422] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5191.951075] Lustre: lustre-MDT0001: Aborting client recovery [ 5192.318188] loop2: detected capacity change from 0 to 204800000 [ 5192.353730] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5192.409312] Lustre: 279736:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5192.414238] Lustre: 279736:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5192.420190] Lustre: 279736:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5192.425703] Lustre: 279736:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5192.431967] Lustre: 279736:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5192.437304] Lustre: 279736:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5192.443338] Lustre: 279736:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5192.503957] Lustre: Found index 0 for lustre-OST0000, updating log [ 5192.522789] Lustre: 279754:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5192.560956] Lustre: lustre-OST0000: Aborting recovery [ 5192.918642] loop3: detected capacity change from 0 to 204800000 [ 5192.969025] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5192.997659] Lustre: 279819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5193.004748] Lustre: 279819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5193.009572] Lustre: 279819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5193.014108] Lustre: 279819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5193.018643] Lustre: 279819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5193.022902] Lustre: 279819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5193.027595] Lustre: 279819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5193.045214] Lustre: 279824:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5193.064194] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5193.067196] Lustre: Skipped 1 previous similar message [ 5193.073765] Lustre: lustre-OST0001: Aborting recovery [ 5198.316076] LustreError: 279599:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 5201.379873] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5201.380348] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5201.380969] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 5201.387280] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 5201.401466] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 5201.405409] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 5203.728825] LustreError: 279563:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 289us: rc = -2 [ 5216.736445] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5216.736941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5216.742108] Lustre: Skipped 2 previous similar messages [ 5216.744058] Lustre: Skipped 1 previous similar message [ 5221.472835] Lustre: server umount lustre-MDT0000 complete [ 5221.826847] LustreError: 279594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408215 with bad export cookie 502389755551091962 [ 5221.828458] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5221.832191] LustreError: 279594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5221.855544] LustreError: 279562:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5221.855718] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5221.864657] LustreError: 279562:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5221.872171] Lustre: Skipped 1 previous similar message [ 5221.873085] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5221.881782] Lustre: Skipped 3 previous similar messages [ 5226.975464] LustreError: 279562:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5226.975514] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5226.982121] LustreError: 279562:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5226.987556] Lustre: Skipped 1 previous similar message [ 5228.154943] Lustre: server umount lustre-MDT0001 complete [ 5234.573217] Lustre: server umount lustre-OST0000 complete [ 5241.055795] Lustre: server umount lustre-OST0001 complete [ 5243.871639] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 16:57:16 (1784408236) [ 5254.108315] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 5257.506628] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5271.208417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5271.322212] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5271.335235] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5271.373841] Lustre: lustre-MDT0000: new disk, initializing [ 5271.398587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5271.406948] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5273.086321] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5277.839082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5277.893903] Lustre: 283888:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5277.923312] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5277.926499] Lustre: Skipped 1 previous similar message [ 5277.965377] Lustre: lustre-MDT0001: new disk, initializing [ 5278.004553] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5278.019348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5278.023413] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5279.493587] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5283.567071] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5285.705477] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5288.278288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5288.397834] Lustre: lustre-OST0000: new disk, initializing [ 5288.400546] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5288.404247] Lustre: 285857:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5288.441396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5290.668128] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5294.734244] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5296.124632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5296.129327] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5296.166163] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5302.332560] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5307.510226] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5316.037677] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5323.645949] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 5367.777920] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5367.778030] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5367.781625] LustreError: Skipped 1 previous similar message [ 5367.786222] Lustre: Skipped 1 previous similar message [ 5367.788948] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5367.792707] Lustre: Skipped 1 previous similar message [ 5372.898376] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5372.900522] Lustre: Skipped 1 previous similar message [ 5373.586381] Lustre: server umount lustre-OST0000 complete [ 5375.455809] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5375.458703] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5383.136318] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5383.140162] Lustre: Skipped 4 previous similar messages [ 5389.279110] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5389.342303] Lustre: server umount lustre-MDT0000 complete [ 5390.664265] LustreError: 287717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408384 with bad export cookie 502389755551094685 [ 5390.665704] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5390.668260] LustreError: 287717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5390.793391] Lustre: server umount lustre-MDT0001 complete [ 5393.760242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5395.185794] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5396.832124] Lustre: server umount MGS complete [ 5400.735892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5400.915446] LustreError: 289861:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5400.924270] LustreError: 289874:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5400.949370] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5402.368370] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5405.286441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5406.716897] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5409.152517] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5410.850534] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5412.977416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5414.916356] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5417.224686] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5440.992898] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5440.994360] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5440.995160] LustreError: Skipped 1 previous similar message [ 5440.998909] Lustre: Skipped 1 previous similar message [ 5441.000489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5441.003174] Lustre: Skipped 3 previous similar messages [ 5443.853914] Lustre: server umount lustre-OST0000 complete [ 5451.219382] Lustre: server umount lustre-MDT0000 complete [ 5451.231834] LustreError: 289862:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5451.237794] LustreError: 289862:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5452.485170] LustreError: 289843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408446 with bad export cookie 502389755551096512 [ 5452.487504] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5452.489817] LustreError: 289843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5452.603254] Lustre: server umount lustre-MDT0001 complete [ 5456.209134] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 5457.208481] Key type lgssc unregistered [ 5457.340384] LNet: 293820:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5457.344034] LNetError: 293820:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5457.351438] LNet: Removed LNI 192.168.203.117@tcp [ 5457.687889] Key type .llcrypt unregistered [ 5457.689417] Key type ._llcrypt unregistered [ 5470.494234] Key type ._llcrypt registered [ 5470.495675] Key type .llcrypt registered [ 5470.539304] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 5476.531332] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5477.052315] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5477.082979] alg: No test for adler32 (adler32-zlib) [ 5477.930687] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 5478.028936] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 5479.615107] Key type lgssc registered [ 5479.927311] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5491.517444] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5491.523060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5492.613707] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5492.630262] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5492.671881] Lustre: lustre-MDT0000: new disk, initializing [ 5492.702421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5492.709445] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5494.122543] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5498.397060] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5498.442238] Lustre: 297762:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5498.470659] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5498.473107] Lustre: Skipped 1 previous similar message [ 5498.513312] Lustre: lustre-MDT0001: new disk, initializing [ 5498.548774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5498.561385] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5498.565038] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5499.926321] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5503.651052] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5505.565992] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5507.750020] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5507.850773] Lustre: lustre-OST0000: new disk, initializing [ 5507.852345] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5507.854822] Lustre: 299728:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5507.878052] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5509.648647] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5510.390181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5510.393942] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5510.426144] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5513.183588] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5516.615652] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5520.405287] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5527.467182] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5566.945107] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5566.946272] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5566.948245] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5566.952161] Lustre: Skipped 1 previous similar message [ 5572.065753] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5572.068009] Lustre: Skipped 2 previous similar messages [ 5577.186072] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5577.188936] Lustre: Skipped 1 previous similar message [ 5580.255093] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5580.304209] Lustre: server umount lustre-OST0000 complete [ 5582.305075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5582.305983] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5582.308301] Lustre: Skipped 1 previous similar message [ 5587.424096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5587.426613] Lustre: Skipped 2 previous similar messages [ 5587.605904] Lustre: server umount lustre-MDT0000 complete [ 5588.777396] LustreError: 297798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408582 with bad export cookie 13821894677801064297 [ 5588.780695] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5588.781814] LustreError: 297798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5588.896671] Lustre: server umount lustre-MDT0001 complete [ 5591.706038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5593.043121] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5594.605083] Lustre: server umount MGS complete [ 5598.128766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5598.274059] LustreError: 303650:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5598.295560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5599.502896] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5602.061535] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5603.362278] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5605.532582] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5607.296346] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5609.550514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5609.673421] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5609.676625] Lustre: Skipped 1 previous similar message [ 5611.608285] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5613.986716] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5644.769939] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5644.770237] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5644.772052] LustreError: Skipped 1 previous similar message [ 5644.772512] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5644.776440] Lustre: Skipped 1 previous similar message [ 5650.448638] Lustre: server umount lustre-OST0000 complete [ 5653.471751] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5653.473833] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5657.818864] Lustre: server umount lustre-MDT0000 complete [ 5659.130103] LustreError: 306383:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408653 with bad export cookie 13821894677801066110 [ 5659.134324] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5659.135173] LustreError: 306383:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5659.262178] Lustre: server umount lustre-MDT0001 complete [ 5662.900336] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 5663.879927] Key type lgssc unregistered [ 5664.013340] LNet: 307599:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5664.016727] LNetError: 307599:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5664.027311] LNet: Removed LNI 192.168.203.117@tcp [ 5664.319243] Key type .llcrypt unregistered [ 5664.320732] Key type ._llcrypt unregistered [ 5670.580450] Key type ._llcrypt registered [ 5670.581862] Key type .llcrypt registered [ 5670.614815] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 5676.388458] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5676.690061] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5676.695873] alg: No test for adler32 (adler32-zlib) [ 5677.554386] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 5677.639495] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 5679.215141] Key type lgssc registered [ 5679.620240] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5687.466313] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5692.668939] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5692.675680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5693.770880] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5693.784667] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5693.827066] Lustre: lustre-MDT0000: new disk, initializing [ 5693.849207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5693.856242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5695.272698] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5699.429413] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5699.543878] Lustre: lustre-OST0000: new disk, initializing [ 5699.546472] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 5699.546974] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5699.552685] Lustre: Skipped 1 previous similar message [ 5699.555843] Lustre: 311624:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5699.578709] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5701.574188] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5704.691928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5704.696385] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5704.705818] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5878.147371] Lustre: server umount lustre-MDT0000 complete [ 5879.468431] LustreError: 310810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408873 with bad export cookie 16929296060592474477 [ 5879.474193] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5890.067549] Lustre: server umount lustre-OST0000 complete [ 5897.976016] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 5900.996161] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 5912.409464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5912.512605] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5912.523492] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5912.562798] Lustre: lustre-MDT0000: new disk, initializing [ 5912.584450] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5912.590745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5913.947706] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5917.915440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5917.953559] Lustre: 315846:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5917.974638] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5917.977123] Lustre: Skipped 1 previous similar message [ 5918.011726] Lustre: lustre-MDT0001: new disk, initializing [ 5918.039464] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5918.049556] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5918.054625] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5919.248374] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5922.844517] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5924.594379] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5926.733198] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5926.826445] Lustre: lustre-OST0000: new disk, initializing [ 5926.827765] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5926.829788] Lustre: 317815:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5926.853717] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5927.124590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5927.127676] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5927.158654] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5928.635430] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5932.186787] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5933.511467] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5933.577232] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5934.813391] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5934.881139] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5937.556884] Lustre: server umount lustre-OST0000 complete [ 5942.752549] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5942.753222] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5942.756206] Lustre: Skipped 1 previous similar message [ 5947.872606] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5947.874543] Lustre: Skipped 2 previous similar messages [ 5948.920569] Lustre: server umount lustre-MDT0000 complete [ 5950.202190] LustreError: 315839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784408944 with bad export cookie 16929296060595415268 [ 5950.205157] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5950.207166] LustreError: 315839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5960.724588] Lustre: server umount lustre-MDT0001 complete [ 5961.959782] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5962.523263] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:09:14 (1784408954) [ 5964.805685] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5965.008437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5966.381155] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5969.108048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5970.577632] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5972.903588] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5974.635307] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5976.758955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5978.616864] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5980.917113] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5983.139611] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5983.216101] Lustre: lustre-OST0001: new disk, initializing [ 5983.218532] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5983.221842] Lustre: 322960:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5983.223627] Lustre: lustre-OST0001: Not available for connect from 0@lo (not set up) [ 5983.263330] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5983.266382] Lustre: Skipped 2 previous similar messages [ 5985.035993] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 5988.328541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5988.331678] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5988.332597] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5989.350907] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5989.355828] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5990.369521] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5990.373483] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5992.417671] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5992.421920] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5994.066873] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5994.128068] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5995.844287] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff996bc55d4000.ost_server_uuid 50 [ 5996.333658] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff996bc55d4000.ost_server_uuid in FULL state after 0 sec [ 5996.513241] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5996.515807] LustreError: 322972:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5998.031825] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff996bc55d4000.ost_server_uuid 50 [ 5998.523255] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff996bc55d4000.ost_server_uuid in FULL state after 0 sec [ 5999.594417] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6013.214780] Lustre: lustre-OST0001: Not available for connect from 192.168.203.17@tcp (stopping) [ 6014.947290] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6014.948274] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6014.949432] LustreError: Skipped 1 previous similar message [ 6014.954028] Lustre: Skipped 1 previous similar message [ 6016.191335] Lustre: server umount lustre-OST0001 complete [ 6020.065241] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6020.065885] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6020.068113] LustreError: 322504:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6020.068120] LustreError: 322504:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6020.078566] Lustre: Skipped 1 previous similar message [ 6023.760782] Lustre: server umount lustre-OST0000 complete [ 6025.185455] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6025.186815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6025.190178] Lustre: Skipped 1 previous similar message [ 6025.191800] Lustre: Skipped 4 previous similar messages [ 6031.067537] Lustre: server umount lustre-MDT0000 complete [ 6032.263995] LustreError: 320108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784409026 with bad export cookie 16929296060595416136 [ 6032.267140] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6032.268353] LustreError: 320108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6032.391574] Lustre: server umount lustre-MDT0001 complete [ 6035.862551] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 6036.882276] Key type lgssc unregistered [ 6037.013412] LNet: 325642:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6037.017839] LNetError: 325642:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6037.025292] LNet: Removed LNI 192.168.203.117@tcp [ 6037.306373] Key type .llcrypt unregistered [ 6037.307450] Key type ._llcrypt unregistered [ 6046.279643] Key type ._llcrypt registered [ 6046.280680] Key type .llcrypt registered [ 6046.320081] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 6046.729656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6046.751018] alg: No test for adler32 (adler32-zlib) [ 6047.604675] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 6047.695466] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 6049.279174] Key type lgssc registered [ 6049.634098] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6052.082555] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6052.087938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6053.219353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6054.471970] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6056.668326] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6058.771790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6058.900055] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6060.106527] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6063.202574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6063.337409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6064.355469] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6064.359771] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 6064.362411] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6065.383322] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6067.812408] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6068.450021] LustreError: 328899:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6068.456289] LustreError: 328899:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6069.909360] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6069.985727] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6071.954178] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6074.380184] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6075.374417] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6076.908456] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 6076.983793] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6157.281670] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6157.284069] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6157.289073] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6157.291021] Lustre: Skipped 1 previous similar message [ 6158.970190] Lustre: server umount lustre-OST0001 complete [ 6160.864233] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6160.867110] LustreError: Skipped 1 previous similar message [ 6160.868867] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6160.872953] Lustre: Skipped 1 previous similar message [ 6160.874578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6161.887964] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6161.891064] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6161.896627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6162.401160] LustreError: 328721:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6162.406076] LustreError: 328721:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6166.670205] Lustre: server umount lustre-OST0000 complete [ 6171.615992] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6171.618713] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6171.625338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6171.628890] Lustre: Skipped 2 previous similar messages [ 6177.760127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6177.762012] Lustre: Skipped 3 previous similar messages [ 6182.367074] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6182.432297] Lustre: server umount lustre-MDT0000 complete [ 6182.880243] LustreError: 327040:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6182.888686] LustreError: 327040:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6183.695214] LustreError: 327971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784409177 with bad export cookie 11227115306434994929 [ 6183.698664] LustreError: 327971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6183.699023] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6183.823416] Lustre: server umount lustre-MDT0001 complete [ 6187.601384] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 6188.661364] Key type lgssc unregistered [ 6188.793871] LNet: 331974:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6188.797324] LNetError: 331974:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6188.807378] LNet: Removed LNI 192.168.203.117@tcp [ 6189.164338] Key type .llcrypt unregistered [ 6189.166299] Key type ._llcrypt unregistered [ 6206.424219] Key type ._llcrypt registered [ 6206.425455] Key type .llcrypt registered [ 6206.460853] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 6206.819533] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6206.881050] alg: No test for adler32 (adler32-zlib) [ 6207.735628] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 6207.822611] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 6209.399211] Key type lgssc registered [ 6209.801752] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6212.913218] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6212.918652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6214.043909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6215.220895] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6218.060967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6218.212673] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6219.635794] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6220.625281] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6222.677785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6222.809901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6223.843050] LustreError: 335731:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6223.845196] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 6223.847415] LustreError: 335731:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6224.597741] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6227.242279] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6227.309781] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6228.338044] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 6229.194405] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6233.164064] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6238.803591] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6243.812040] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6243.812933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6243.816957] Lustre: Skipped 3 previous similar messages [ 6243.818589] Lustre: Skipped 2 previous similar messages [ 6243.822202] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6248.282388] Lustre: server umount lustre-MDT0000 complete [ 6248.930238] LustreError: 334244:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6248.937839] LustreError: 334244:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6249.534531] LustreError: 334982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784409243 with bad export cookie 18105487356082834429 [ 6249.538435] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6249.539421] LustreError: 334982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6249.644531] Lustre: server umount lustre-MDT0001 complete [ 6260.225212] Lustre: server umount lustre-OST0000 complete [ 6271.647125] Lustre: server umount lustre-OST0001 complete [ 6276.563947] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 6279.855058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6279.974675] LustreError: 339501:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6279.990640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6281.287706] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6283.950446] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6285.308368] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6286.259418] Lustre: 340642:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6288.430958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6288.570135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6288.574236] Lustre: Skipped 1 previous similar message [ 6289.571144] LustreError: 340996:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6289.573960] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 6290.453253] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6292.983494] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6294.130891] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 6294.835378] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6297.904120] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6299.072925] Lustre: 342514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6304.740149] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6304.740526] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6304.745581] Lustre: Skipped 1 previous similar message [ 6304.745764] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6304.745774] Lustre: Skipped 1 previous similar message [ 6306.145682] Lustre: server umount lustre-MDT0000 complete [ 6307.281354] LustreError: 339482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784409301 with bad export cookie 18105487356082836088 [ 6307.286444] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6307.409787] Lustre: server umount lustre-MDT0001 complete [ 6319.041165] Lustre: server umount lustre-OST0000 complete [ 6330.599421] Lustre: server umount lustre-OST0001 complete [ 6331.827570] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6332.391220] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6332.975542] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 17:15:25 (1784409325) [ 6340.045554] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 6343.072305] loop0: detected capacity change from 0 to 1099511627776 [ 6343.213708] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6343.217290] *** VALIDATE xfs *** [ 6343.223491] XFS (loop0): Mounting V5 Filesystem [ 6343.232989] XFS (loop0): Ending clean mount [ 6343.680096] loop1: detected capacity change from 0 to 18691697672192 [ 6364.694132] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 6367.887633] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 6379.867678] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6379.962618] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6379.973124] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6380.012203] Lustre: lustre-MDT0000: new disk, initializing [ 6380.033206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6380.036144] Lustre: Skipped 1 previous similar message [ 6380.042318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6381.366869] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6385.568950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6385.618283] Lustre: 347624:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6385.644137] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6385.646433] Lustre: Skipped 1 previous similar message [ 6385.694053] Lustre: lustre-MDT0001: new disk, initializing [ 6385.740871] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6385.745174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6387.164044] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6390.917209] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6392.781131] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6395.110809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6395.222894] Lustre: lustre-OST0000: new disk, initializing [ 6395.224584] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6395.227704] Lustre: 349593:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6396.859638] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6396.862708] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6396.910762] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6397.337067] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6401.112844] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6402.431739] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6402.503252] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6403.847591] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6403.919880] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6407.137234] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6407.138553] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6407.139025] LustreError: Skipped 1 previous similar message [ 6407.141649] Lustre: Skipped 2 previous similar messages [ 6407.142304] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6407.144677] Lustre: Skipped 4 previous similar messages [ 6410.643106] Lustre: server umount lustre-OST0000 complete [ 6412.256776] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6412.257739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6412.259376] Lustre: Skipped 2 previous similar messages [ 6412.261468] Lustre: Skipped 1 previous similar message [ 6417.376276] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6417.377818] Lustre: Skipped 1 previous similar message [ 6418.010013] Lustre: server umount lustre-MDT0000 complete [ 6419.233328] LustreError: 348421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784409413 with bad export cookie 18105487356082837621 [ 6419.236265] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6419.237062] LustreError: 348421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 6419.358142] Lustre: server umount lustre-MDT0001 complete [ 6421.932561] XFS (loop0): Unmounting Filesystem [ 6439.153362] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 17:17:11 (1784409431) [ 6441.495910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6441.650065] LustreError: 352165:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6441.673215] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6441.675308] Lustre: Skipped 2 previous similar messages [ 6442.915670] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6445.577733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6447.105889] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6449.366926] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6451.083947] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6453.208121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6455.181903] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6457.443711] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6471.135866] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6471.138351] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6471.141991] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6476.890246] Lustre: server umount lustre-OST0000 complete [ 6484.374837] Lustre: server umount lustre-MDT0000 complete [ 6485.602243] LustreError: 352147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784409479 with bad export cookie 18105487356082838503 [ 6485.605894] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6485.608918] LustreError: 352147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6485.730930] Lustre: server umount lustre-MDT0001 complete [ 6489.239207] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 6490.242501] Key type lgssc unregistered [ 6490.372328] LNet: 355814:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6490.374577] LNetError: 355814:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6490.380243] LNet: Removed LNI 192.168.203.117@tcp [ 6490.685104] Key type .llcrypt unregistered [ 6490.685989] Key type ._llcrypt unregistered [ 6500.493597] Key type ._llcrypt registered [ 6500.495411] Key type .llcrypt registered [ 6500.536233] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 6500.856763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6500.887031] alg: No test for adler32 (adler32-zlib) [ 6501.738441] Lustre: Lustre: Build Version: 2.17.54_174_g53e1600 [ 6501.826380] LNet: Added LNI 192.168.203.117@tcp [8/256/0/180] [ 6503.407116] Key type lgssc registered [ 6503.742170] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6506.260062] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6506.268041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6507.410393] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6508.674414] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6511.448324] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6511.580244] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6513.089114] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6515.571506] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6517.365166] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6519.476036] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6519.592908] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6521.413549] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6523.693630] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6537.184198] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6537.187613] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6537.191791] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6538.212687] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6538.212901] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6538.221035] Lustre: Skipped 1 previous similar message [ 6539.551852] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.17@tcp (stopping) [ 6543.259892] Lustre: server umount lustre-MDT0001 complete [ 6543.329521] LustreError: 357356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6543.336967] LustreError: 357356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6544.664147] LustreError: 359570:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6547.056817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6547.101375] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6547.104116] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6547.121031] Lustre: 357343:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6547.271116] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6548.852641] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6552.549858] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6556.134112] LustreError: 361186:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6556.137356] LustreError: 361186:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6556.140388] LustreError: 357348:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6556.157308] LustreError: 361189:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.117@tcp: cfg command failed: rc = -17 [ 6556.160410] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6556.160410] [ 6851.870542] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.17@tcp (stopping) [ 6854.624667] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6854.625263] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6856.984603] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.17@tcp (stopping) [ 6856.987393] Lustre: Skipped 2 previous similar messages [ 6857.498084] Lustre: server umount lustre-MDT0001 complete [ 6859.746606] LustreError: 360004:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6859.752370] LustreError: 360004:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6860.929440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6860.964672] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6860.966643] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6860.981346] Lustre: 357345:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6861.092330] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6862.485638] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 6866.411431] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6871.015494] LustreError: 373863:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6871.018309] LustreError: 373863:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6871.022894] LustreError: 357348:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6871.025062] LustreError: 357348:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6871.045706] LustreError: 373866:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6871.049256] LustreError: 373866:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6871.052357] LustreError: 373866:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.117@tcp: cfg command failed: rc = -17 [ 6871.056595] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.117@tcp [ 6871.056595] [ 7164.191453] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.17@tcp (stopping) [ 7168.483795] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7168.484743] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7168.490995] Lustre: Skipped 1 previous similar message [ 7170.136988] Lustre: server umount lustre-MDT0001 complete [ 7173.600894] LustreError: 357356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7173.607808] LustreError: 357356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7174.204469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7174.256963] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7174.259653] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7174.280504] Lustre: 357345:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7174.412916] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 7175.831243] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 7179.757096] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7181.290506] LustreError: 386237:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7181.295475] LustreError: 386237:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7181.301409] LustreError: 357348:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7181.305446] LustreError: 357348:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7181.332648] LustreError: 386246:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 7181.338462] LustreError: 386246:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.117@tcp: cfg command failed: rc = -114 [ 7181.342473] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 7181.342473] [ 7492.066151] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7492.066463] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7492.067514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7492.067520] Lustre: Skipped 3 previous similar messages [ 7492.080156] Lustre: Skipped 3 previous similar messages [ 7504.351171] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7504.414159] Lustre: server umount lustre-MDT0000 complete [ 7505.640547] LustreError: 357336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784410499 with bad export cookie 11095463532423789251 [ 7505.643681] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7505.646371] LustreError: 357336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7507.424368] LustreError: 357352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7507.425049] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7511.889162] Lustre: server umount lustre-MDT0001 complete [ 7513.227340] Lustre: server umount lustre-OST0000 complete [ 7516.190562] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 7519.749608] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 7532.283286] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 17:35:24 (1784410524) [ 7534.751103] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7534.854549] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7534.865221] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7534.905264] Lustre: lustre-MDT0000: new disk, initializing [ 7534.936600] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7534.944996] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7536.589281] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 7541.268533] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7541.326133] Lustre: 400599:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7541.366539] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7541.369569] Lustre: Skipped 1 previous similar message [ 7541.412972] Lustre: lustre-MDT0001: new disk, initializing [ 7541.462539] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7541.485717] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7541.493524] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7543.197375] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 7547.377877] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7549.416851] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7551.770441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7551.881112] Lustre: lustre-OST0000: new disk, initializing [ 7551.883672] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7551.887876] Lustre: 402567:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7551.926301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7553.960301] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_default_debug -1 all [ 7558.090856] Lustre: DEBUG MARKER: oleg317-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7560.200955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7560.207043] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7560.254736] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7567.328413] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7567.331739] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7567.337193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7567.339853] Lustre: Skipped 10 previous similar messages [ 7572.406919] Lustre: server umount lustre-MDT0000 complete [ 7573.997087] LustreError: 400592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784410567 with bad export cookie 11095463532423791190 [ 7574.005392] LustreError: 400592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7574.005458] LustreError: MGC192.168.203.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7574.144616] Lustre: server umount lustre-MDT0001 complete [ 7585.893921] Lustre: server umount lustre-OST0000 complete [ 7596.033912] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing set_hostid [ 7599.130741] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing load_modules_local [ 7609.317886] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7370 sec ========== 17:36:41 (1784410601) [ 7610.118941] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 17:36:42 (1784410602) === [ 7611.660682] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 17:36:43 (1784410603) === [ 7626.546637] Lustre: DEBUG MARKER: oleg317-server.virtnet: executing unload_modules_local [ 7627.571975] Key type lgssc unregistered [ 7627.715580] LNet: 408923:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7627.719482] LNetError: 408923:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7627.732343] LNet: Removed LNI 192.168.203.117@tcp [ 7628.093132] Key type .llcrypt unregistered [ 7628.095489] Key type ._llcrypt unregistered