[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 447334403 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002283] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.005008] kvm-guest: setup PV IPIs [ 0.008525] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009010] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010006] pid_max: default: 32768 minimum: 301 [ 0.012034] LSM: Security Framework initializing [ 0.013034] Yama: becoming mindful. [ 0.014028] SELinux: Initializing. [ 0.015040] *** VALIDATE selinux *** [ 0.024107] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028646] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029102] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030067] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031061] *** VALIDATE tmpfs *** [ 0.032464] *** VALIDATE proc *** [ 0.033219] *** VALIDATE cgroup *** [ 0.034006] *** VALIDATE cgroup2 *** [ 0.035266] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037036] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039024] Spectre V2 : User space: Vulnerable [ 0.040004] Speculative Store Bypass: Vulnerable [ 0.042650] debug: unmapping init [mem 0xffffffffb8e59000-0xffffffffb8e60fff] [ 0.044125] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045407] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046010] ... version: 2 [ 0.046897] ... bit width: 48 [ 0.047006] ... generic registers: 4 [ 0.048007] ... value mask: 0000ffffffffffff [ 0.049006] ... max period: 00007fffffffffff [ 0.050006] ... fixed-purpose events: 3 [ 0.051008] ... event mask: 000000070000000f [ 0.052191] rcu: Hierarchical SRCU implementation. [ 0.054200] smp: Bringing up secondary CPUs ... [ 0.055429] x86: Booting SMP configuration: [ 0.056014] .... node #0, CPUs: #1 #2 #3 [ 0.058874] smp: Brought up 1 node, 4 CPUs [ 0.059948] smpboot: Max logical packages: 1 [ 0.060006] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.177167] node 0 deferred pages initialised in 116ms [ 0.180377] devtmpfs: initialized [ 0.181228] x86/mm: Memory block size: 128MB [ 0.183997] gcov: version magic: 0x41383552 [ 0.186170] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.187074] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.188203] pinctrl core: initialized pinctrl subsystem [ 0.189166] [ 0.189643] ************************************************************* [ 0.190008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.191006] ** ** [ 0.192007] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.193007] ** ** [ 0.194008] ** This means that this kernel is built to expose internal ** [ 0.195018] ** IOMMU data structures, which may compromise security on ** [ 0.196012] ** your system. ** [ 0.197008] ** ** [ 0.198008] ** If you see this message and you are not debugging the ** [ 0.199008] ** kernel, report this immediately to your vendor! ** [ 0.200008] ** ** [ 0.201008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.202009] ************************************************************* [ 0.203672] NET: Registered protocol family 16 [ 0.204426] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.205042] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.206028] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.207340] cpuidle: using governor menu [ 0.208000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.210398] PCI: Using configuration type 1 for base access [ 0.212093] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.219105] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.220017] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.221162] cryptd: max_cpu_qlen set to 1000 [ 0.223067] ACPI: Added _OSI(Module Device) [ 0.224007] ACPI: Added _OSI(Processor Device) [ 0.225016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.226000] ACPI: Added _OSI(Processor Aggregator Device) [ 0.229284] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.234453] ACPI: Interpreter enabled [ 0.235043] ACPI: PM: (supports S0 S3 S4 S5) [ 0.236000] ACPI: Using IOAPIC for interrupt routing [ 0.237082] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.239313] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.246647] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.248017] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.250015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.254063] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.257034] acpiphp: Slot [2] registered [ 0.257980] acpiphp: Slot [5] registered [ 0.259092] acpiphp: Slot [6] registered [ 0.260052] acpiphp: Slot [7] registered [ 0.260887] acpiphp: Slot [8] registered [ 0.262056] acpiphp: Slot [9] registered [ 0.262939] acpiphp: Slot [10] registered [ 0.264052] acpiphp: Slot [3] registered [ 0.264913] acpiphp: Slot [4] registered [ 0.266070] acpiphp: Slot [11] registered [ 0.267033] acpiphp: Slot [12] registered [ 0.268065] acpiphp: Slot [13] registered [ 0.268840] acpiphp: Slot [14] registered [ 0.270064] acpiphp: Slot [15] registered [ 0.271078] acpiphp: Slot [16] registered [ 0.273080] acpiphp: Slot [17] registered [ 0.275099] acpiphp: Slot [18] registered [ 0.276076] acpiphp: Slot [19] registered [ 0.277080] acpiphp: Slot [20] registered [ 0.279076] acpiphp: Slot [21] registered [ 0.280068] acpiphp: Slot [22] registered [ 0.282064] acpiphp: Slot [23] registered [ 0.283063] acpiphp: Slot [24] registered [ 0.284102] acpiphp: Slot [25] registered [ 0.285065] acpiphp: Slot [26] registered [ 0.287065] acpiphp: Slot [27] registered [ 0.288062] acpiphp: Slot [28] registered [ 0.289078] acpiphp: Slot [29] registered [ 0.290065] acpiphp: Slot [30] registered [ 0.292075] acpiphp: Slot [31] registered [ 0.293054] PCI host bridge to bus 0000:00 [ 0.294012] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.296011] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.298011] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.300010] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.302011] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.305014] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.306188] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.308952] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.312121] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.321726] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.326069] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.328010] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.330010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.332014] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.334611] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.337625] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.339028] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.341560] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.345009] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.357855] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.362011] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.368348] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.373008] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.379020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.392011] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.400323] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.406011] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.412012] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.425013] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.433576] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.438008] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.443011] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.458011] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.466258] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.471010] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.476009] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.490901] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.498443] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.503864] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.509011] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.522011] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.530551] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.536014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.542015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.554011] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.562027] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.564262] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.566240] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.567191] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.569108] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.572033] iommu: Default domain type: Passthrough [ 0.574254] SCSI subsystem initialized [ 0.575087] ACPI: bus type USB registered [ 0.577087] usbcore: registered new interface driver usbfs [ 0.580085] usbcore: registered new interface driver hub [ 0.582075] usbcore: registered new device driver usb [ 0.584211] pps_core: LinuxPPS API ver. 1 registered [ 0.586010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.589041] PTP clock support registered [ 0.591113] EDAC MC: Ver: 3.0.0 [ 0.592539] PCI: Using ACPI for IRQ routing [ 0.593670] NetLabel: Initializing [ 0.595008] NetLabel: domain hash size = 128 [ 0.597009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.598068] NetLabel: unlabeled traffic allowed by default [ 0.601105] vgaarb: loaded [ 0.602262] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.604010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.610005] clocksource: Switched to clocksource kvm-clock [ 0.705045] VFS: Disk quotas dquot_6.6.0 [ 0.706040] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.707675] *** VALIDATE ramfs *** [ 0.708426] *** VALIDATE hugetlbfs *** [ 0.709605] pnp: PnP ACPI init [ 0.711469] pnp: PnP ACPI: found 6 devices [ 0.724216] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.726661] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.728263] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.729614] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.731613] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.733571] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.735415] NET: Registered protocol family 2 [ 0.737428] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.740942] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.743542] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.747808] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.750634] TCP: Hash tables configured (established 65536 bind 65536) [ 0.752693] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.755268] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.757562] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.759423] NET: Registered protocol family 1 [ 0.760940] RPC: Registered named UNIX socket transport module. [ 0.762282] RPC: Registered udp transport module. [ 0.763599] RPC: Registered tcp transport module. [ 0.765019] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.766729] NET: Registered protocol family 44 [ 0.767858] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.769159] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.770377] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.771788] PCI: CLS 0 bytes, default 64 [ 0.772772] Unpacking initramfs... [ 2.093050] debug: unmapping init [mem 0xffff9a603cc54000-0xffff9a603ffbffff] [ 2.095830] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.097719] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.099699] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.525394] Initialise system trusted keyrings [ 2.526470] Key type blacklist registered [ 2.527620] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.533462] zbud: loaded [ 2.535801] *** VALIDATE nfs *** [ 2.536673] *** VALIDATE nfs4 *** [ 2.538082] pstore: using deflate compression [ 2.540196] Platform Keyring initialized [ 2.614353] NET: Registered protocol family 38 [ 2.616068] Key type asymmetric registered [ 2.617449] Asymmetric key parser 'x509' registered [ 2.619349] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.622648] io scheduler mq-deadline registered [ 2.624535] io scheduler kyber registered [ 2.626050] io scheduler bfq registered [ 2.627414] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.629411] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.632341] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.636753] ACPI: Power Button [PWRF] [ 2.643742] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.649863] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.669336] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.677036] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.699040] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.727379] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.754513] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.760783] Non-volatile memory driver v1.3 [ 2.762445] Linux agpgart interface v0.103 [ 2.786578] virtio_blk virtio1: [vda] 139288 512-byte logical blocks (71.3 MB/68.0 MiB) [ 2.789106] vda: detected capacity change from 0 to 71315456 [ 2.804676] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.806816] vdb: detected capacity change from 0 to 1073741824 [ 2.825877] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.828544] vdc: detected capacity change from 0 to 2621440000 [ 2.844562] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.847436] vdd: detected capacity change from 0 to 2621440000 [ 2.864257] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.866097] vde: detected capacity change from 0 to 4294967296 [ 2.880314] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.883355] vdf: detected capacity change from 0 to 4294967296 [ 2.889688] libphy: Fixed MDIO Bus: probed [ 2.893781] usbcore: registered new interface driver usbserial_generic [ 2.895713] usbserial: USB Serial support registered for generic [ 2.897228] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.901081] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.902687] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.904636] mousedev: PS/2 mouse device common for all mice [ 2.908431] rtc_cmos 00:05: RTC can wake from S4 [ 2.911449] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.915220] rtc_cmos 00:05: registered as rtc0 [ 2.917550] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.918227] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.922740] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.924456] intel_pstate: CPU model not supported [ 2.929033] hid: raw HID events driver (C) Jiri Kosina [ 2.930625] usbcore: registered new interface driver usbhid [ 2.932640] usbhid: USB HID core driver [ 2.933962] drop_monitor: Initializing network drop monitor service [ 2.936166] Initializing XFRM netlink socket [ 2.937567] NET: Registered protocol family 10 [ 2.939863] Segment Routing with IPv6 [ 2.941336] NET: Registered protocol family 17 [ 2.943486] mpls_gso: MPLS GSO support [ 2.948574] RAS: Correctable Errors collector initialized. [ 2.950048] AVX version of gcm_enc/dec engaged. [ 2.951274] AES CTR mode by8 optimization enabled [ 3.010208] sched_clock: Marking stable (3010183661, 0)->(3791639340, -781455679) [ 3.014186] registered taskstats version 1 [ 3.016604] Loading compiled-in X.509 certificates [ 3.018890] zswap: loaded using pool lzo/zbud [ 3.037087] Key type big_key registered [ 3.047232] Key type encrypted registered [ 3.048626] ima: No TPM chip found, activating TPM-bypass! [ 3.050293] ima: Allocated hash algorithm: sha1 [ 3.051828] ima: No architecture policies found [ 3.053400] evm: Initialising EVM extended attributes: [ 3.054864] evm: security.selinux [ 3.055646] evm: security.ima [ 3.056485] evm: security.capability [ 3.057478] evm: HMAC attrs: 0x1 [ 3.059422] rtc_cmos 00:05: setting system clock to 2026-06-17 08:13:49 UTC (1781684029) [ 3.065133] debug: unmapping init [mem 0xffffffffb9e03000-0xffffffffb9ffffff] [ 3.067781] debug: unmapping init [mem 0xffffffffb8b82000-0xffffffffb8e58fff] [ 3.075103] Write protecting the kernel read-only data: 28672k [ 3.078293] debug: unmapping init [mem 0xffffffffb7203000-0xffffffffb73fffff] [ 3.080785] debug: unmapping init [mem 0xffffffffb7b14000-0xffffffffb7bfffff] [ 3.110178] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.117542] systemd[1]: Detected virtualization kvm. [ 3.118905] systemd[1]: Detected architecture x86-64. [ 3.120636] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.146737] systemd[1]: No hostname configured. [ 3.148542] systemd[1]: Set hostname to . [ 3.150495] random: systemd: uninitialized urandom read (16 bytes read) [ 3.153054] systemd[1]: Initializing machine ID from random generator. [ 3.206376] random: ln: uninitialized urandom read (6 bytes read) [ 3.280874] random: systemd: uninitialized urandom read (16 bytes read) [ 3.283421] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.287213] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.290365] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Swap. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Journal Service... Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.891372] device-mapper: uevent: version 1.0.3 [ 3.892964] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.660777] virtio_net virtio0 ens2: renamed from eth0 [ 4.694093] random: fast init done [ 4.732902] scsi host0: ata_piix [ 4.780725] scsi host1: ata_piix [ 4.781716] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.783430] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.201660] dracut-initqueue[586]: RTNETLINK answers: File exists [ 9.562534] random: crng init done [ 9.564538] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 9.972298] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.017522] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.264593] SELinux: Disabled at runtime. [ 11.328264] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.340724] systemd[1]: Detected virtualization kvm. [ 11.343030] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.870701] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.873643] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.880260] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.884318] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.889212] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.899231] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.907448] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Kernel Socket. Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... [ OK ] Listening on udev Control Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting POSIX Message Queue File System... [ OK ] Reached target Slices. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... Starting udev Coldplug all Devices... [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 12.036526] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. Mounting Huge Pages File System... [ OK ] Reached target Paths. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.346627] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.628142] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.735766] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.868560] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.884376] EDAC sbridge: Ver: 1.1.2 [ 14.271162] Key type dns_resolver registered [ 14.565515] NFS: Registering the id_resolver key type [ 14.567450] Key type id_resolver registered [ 14.569269] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg434-server login: [ 35.583128] libcfs: loading out-of-tree module taints kernel. [ 35.599709] Key type ._llcrypt registered [ 35.601014] Key type .llcrypt registered [ 35.635758] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 42.249919] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 42.746144] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 42.751191] alg: No test for adler32 (adler32-zlib) [ 43.708473] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 43.944468] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 45.551095] Key type lgssc registered [ 46.079323] Lustre: Echo OBD driver; http://www.lustre.org/ [ 51.626438] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 64.307049] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 68.520069] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 68.529284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 69.617668] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 69.628975] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 69.666187] Lustre: lustre-MDT0000: new disk, initializing [ 69.690839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 69.696484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 70.974838] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 76.131082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 76.172976] Lustre: 6475:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 76.186972] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 76.189207] Lustre: Skipped 1 previous similar message [ 76.230844] Lustre: lustre-MDT0001: new disk, initializing [ 76.256711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 76.270782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 76.276221] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 77.829430] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 80.408696] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 84.065669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 84.165948] Lustre: lustre-OST0000: new disk, initializing [ 84.168291] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 84.170624] Lustre: 8414:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 84.190130] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 86.275207] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 89.074652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 89.081589] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 89.107461] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 92.359364] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 92.405778] Lustre: lustre-OST0001: new disk, initializing [ 92.408149] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 92.411269] Lustre: 9487:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 92.430732] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 94.413366] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 96.302080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 96.306385] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 96.324024] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 100.132487] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 103.725629] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 105.657454] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing check_logdir /tmp/testlogs/ [ 107.161926] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing yml_node [ 108.610091] Lustre: DEBUG MARKER: Client: 2.17.54.40 [ 109.446668] Lustre: DEBUG MARKER: MDS: 2.17.54.40 [ 110.329519] Lustre: DEBUG MARKER: OSS: 2.17.54.40 [ 110.909665] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Jun 17 04:15:36 EDT 2026 [ 116.924497] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 117.446565] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 132.064068] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 132.064371] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 132.069301] Lustre: Skipped 3 previous similar messages [ 132.072174] Lustre: Skipped 2 previous similar messages [ 135.335477] Lustre: server umount lustre-MDT0000 complete [ 136.569895] LustreError: 7425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684163 with bad export cookie 10986272201329841200 [ 136.570075] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 136.572924] LustreError: 7425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 137.183643] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 137.183825] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 137.187316] Lustre: Skipped 1 previous similar message [ 137.190233] Lustre: Skipped 1 previous similar message [ 141.215523] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 141.217658] Lustre: Skipped 1 previous similar message [ 142.746750] Lustre: server umount lustre-MDT0001 complete [ 158.175109] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 158.206282] Lustre: server umount lustre-OST0000 complete [ 165.451634] Lustre: server umount lustre-OST0001 complete [ 167.471108] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 170.493244] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 182.401418] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 182.494283] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 182.507946] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 182.542757] Lustre: lustre-MDT0000: new disk, initializing [ 182.567380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 182.574226] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 183.930856] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 187.852714] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 187.891224] Lustre: 15028:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 187.987820] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 189.205343] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 192.694580] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 194.369433] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 196.612940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 196.720102] Lustre: lustre-OST0000: new disk, initializing [ 196.721960] Lustre: Skipped 1 previous similar message [ 196.724432] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 196.727572] Lustre: Skipped 2 previous similar messages [ 196.729761] Lustre: 16996:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 196.762235] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 196.766366] Lustre: Skipped 1 previous similar message [ 198.392344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 198.396031] Lustre: Skipped 1 previous similar message [ 198.398383] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 198.429657] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 198.596094] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 202.126508] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 203.401096] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 203.467587] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 204.665911] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 204.733992] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 208.865463] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 208.866815] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 208.867175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 208.869496] Lustre: Skipped 1 previous similar message [ 208.871858] LustreError: Skipped 1 previous similar message [ 211.402128] Lustre: server umount lustre-OST0000 complete [ 213.471871] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 213.474669] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 213.479580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 213.481320] Lustre: Skipped 1 previous similar message [ 218.712782] Lustre: server umount lustre-MDT0000 complete [ 219.104100] LustreError: 15039:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 219.109511] LustreError: 15039:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 219.904082] LustreError: 15021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684246 with bad export cookie 10986272201329842719 [ 219.906771] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 219.907800] LustreError: 15021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 220.022689] Lustre: server umount lustre-MDT0001 complete [ 221.548467] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:17:27 (1781684247) [ 223.648721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 223.793924] LustreError: 19254:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 223.818703] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 225.095992] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 227.684778] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 228.999463] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 231.204049] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 232.997295] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 235.197541] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 237.180334] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 239.577280] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 251.746880] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 258.527950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 258.530181] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 258.533608] Lustre: Skipped 1 previous similar message [ 258.535925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 258.537610] Lustre: Skipped 1 previous similar message [ 260.823993] Lustre: server umount lustre-MDT0000 complete [ 261.955148] LustreError: 19235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684288 with bad export cookie 10986272201329843601 [ 261.958748] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 261.959597] LustreError: 19235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 262.071978] Lustre: server umount lustre-MDT0001 complete [ 273.486103] Lustre: server umount lustre-OST0000 complete [ 279.701667] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 283.196429] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 283.327982] LustreError: 23647:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 283.343693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 283.345768] Lustre: Skipped 2 previous similar messages [ 284.613339] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 287.305085] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 288.802360] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 289.731938] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 291.945604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 294.061593] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 296.939499] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 297.005687] Lustre: lustre-OST0001: new disk, initializing [ 297.007603] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 297.010382] Lustre: 25960:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 298.747243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 298.750909] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 298.779087] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 298.968453] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 303.452447] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 306.086679] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 306.089700] Lustre: Skipped 1 previous similar message [ 308.191840] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 308.195162] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 308.200311] Lustre: Skipped 2 previous similar messages [ 308.202625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 308.205476] Lustre: Skipped 3 previous similar messages [ 313.816207] Lustre: server umount lustre-MDT0000 complete [ 314.338417] LustreError: 23642:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 314.343132] LustreError: 23642:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 315.104397] LustreError: 23629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684341 with bad export cookie 10986272201329844448 [ 315.108445] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 315.109841] LustreError: 23629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 315.228364] Lustre: server umount lustre-MDT0001 complete [ 326.655989] Lustre: server umount lustre-OST0000 complete [ 338.138300] Lustre: server umount lustre-OST0001 complete [ 343.301436] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 346.792424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 346.937509] LustreError: 28920:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 348.354343] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 351.327801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 351.488457] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 351.490825] Lustre: Skipped 4 previous similar messages [ 353.046836] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 354.153526] Lustre: 30068:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 356.459797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 357.604176] LustreError: 30421:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 358.659534] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 361.566652] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 363.808675] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 367.015436] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 368.204486] Lustre: 31942:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 372.193226] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 372.196152] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 372.197377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 372.203177] Lustre: Skipped 5 previous similar messages [ 372.206038] Lustre: Skipped 5 previous similar messages [ 375.890149] Lustre: server umount lustre-MDT0000 complete [ 377.145463] LustreError: 32140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684403 with bad export cookie 10986272201329845680 [ 377.149219] LustreError: 32140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 377.150312] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 377.269209] Lustre: server umount lustre-MDT0001 complete [ 388.864871] Lustre: server umount lustre-OST0000 complete [ 400.276447] Lustre: server umount lustre-OST0001 complete [ 403.214711] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:20:29 (1781684429) [ 412.523431] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 416.027222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 416.169208] LustreError: 34737:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 416.175337] LustreError: 34737:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 417.469853] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 420.316666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 421.868132] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 422.850458] Lustre: 35886:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 425.015217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 427.048935] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 429.875390] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 431.900900] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 435.036381] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 436.197723] Lustre: 37762:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 440.799767] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 440.802511] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 440.807105] Lustre: Skipped 1 previous similar message [ 440.809722] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 440.811813] Lustre: Skipped 1 previous similar message [ 443.610405] Lustre: server umount lustre-MDT0000 complete [ 444.857424] LustreError: 37766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684471 with bad export cookie 10986272201329846758 [ 444.861246] LustreError: 37766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 444.862142] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 444.977087] Lustre: server umount lustre-MDT0001 complete [ 455.614551] Lustre: server umount lustre-OST0000 complete [ 466.974710] Lustre: server umount lustre-OST0001 complete [ 469.733325] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:21:35 (1781684495) [ 474.468825] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 477.477037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 477.596675] LustreError: 39899:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 477.603426] LustreError: 39899:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 478.794814] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 481.337551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 481.459678] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 481.461665] Lustre: Skipped 7 previous similar messages [ 482.633097] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 483.511979] Lustre: 41049:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 485.557081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 487.602031] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 490.485148] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 492.401879] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 495.365649] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 496.809982] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 499.924958] Lustre: server umount lustre-MDT0000 complete [ 501.153345] LustreError: 42922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684527 with bad export cookie 10986272201329847850 [ 501.157321] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 501.157929] LustreError: 42922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 507.316530] Lustre: server umount lustre-MDT0001 complete [ 510.191472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 511.263105] LustreError: 43775:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 511.265309] LustreError: 43775:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff9a60afc55f80 x1868231155660672/t0(0) o250->MGC192.168.204.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1781684537 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_01.0' uid:0 gid:0 projid:4294967295 [ 511.271921] LustreError: 43775:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 511.391584] LustreError: 3610:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9a60afc54a80 x1868231155662336/t0(0) o250->MGC192.168.204.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 512.867722] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 515.775624] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.299769] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 519.646425] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 530.485266] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 10 sec [ 531.301431] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 532.127445] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 532.642229] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 536.546713] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 536.548192] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 536.551021] Lustre: Skipped 9 previous similar messages [ 539.924030] Lustre: server umount lustre-MDT0000 complete [ 541.036403] LustreError: 40649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684567 with bad export cookie 10986272201329848928 [ 541.039585] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 541.039877] LustreError: 40649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 541.140351] Lustre: server umount lustre-MDT0001 complete [ 552.513219] Lustre: server umount lustre-OST0000 complete [ 563.992989] Lustre: server umount lustre-OST0001 complete [ 566.500598] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:23:12 (1781684592) [ 571.669727] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 575.400829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 575.532210] LustreError: 47537:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 575.538646] LustreError: 47537:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 576.940829] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 579.822474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 581.336055] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 582.332159] Lustre: 48690:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 582.336324] Lustre: 48690:0:(mgs_llog.c:1447:mgs_modify_param()) Skipped 1 previous similar message [ 584.583177] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 586.575844] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 589.448361] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 591.482326] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 594.567823] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 595.973273] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 600.543707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 600.548330] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 600.550178] Lustre: Skipped 14 previous similar messages [ 606.425687] Lustre: server umount lustre-MDT0000 complete [ 607.609320] LustreError: 50572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684634 with bad export cookie 10986272201329849754 [ 607.612170] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 607.612961] LustreError: 50572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 607.720299] Lustre: server umount lustre-MDT0001 complete [ 619.526564] Lustre: server umount lustre-OST0000 complete [ 631.063405] Lustre: server umount lustre-OST0001 complete [ 634.106537] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:24:20 (1781684660) [ 634.906839] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 635.423590] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:24:21 (1781684661) [ 638.633551] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 641.029531] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 647.693433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 647.786080] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 647.821503] Lustre: lustre-MDT0000: new disk, initializing [ 647.849903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 649.155086] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 652.655907] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 654.773143] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 654.861695] Lustre: 54755:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 656.612351] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 656.827422] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 656.834492] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 660.003364] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 660.624151] Lustre: DEBUG MARKER: create test files [ 672.224736] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 672.225052] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 672.231609] Lustre: Skipped 4 previous similar messages [ 675.518092] Lustre: server umount lustre-OST0000 complete [ 676.808892] Lustre: server umount lustre-MDT0000 complete [ 680.341362] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 681.340210] Key type lgssc unregistered [ 681.473685] LNet: 56361:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 681.476985] LNetError: 56361:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 681.489423] LNet: Removed LNI 192.168.204.134@tcp [ 681.826132] Key type .llcrypt unregistered [ 681.827396] Key type ._llcrypt unregistered [ 691.640505] Key type ._llcrypt registered [ 691.641400] Key type .llcrypt registered [ 691.675104] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 692.042865] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 692.049370] alg: No test for adler32 (adler32-zlib) [ 692.900457] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 692.978768] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 694.559131] Key type lgssc registered [ 694.871646] Lustre: Echo OBD driver; http://www.lustre.org/ [ 697.206538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 697.214299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 698.345687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 699.570233] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 701.732703] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 703.773821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 703.874661] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 705.606187] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 707.710988] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 708.003148] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 708.020840] Lustre: 59066:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 708.023162] Lustre: 59066:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 708.025034] Lustre: 59066:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 708.026821] Lustre: 59066:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 708.028814] Lustre: 59066:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 708.030547] Lustre: 59066:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 716.071718] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 717.488949] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 723.424568] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 723.424772] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 723.429255] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 725.388329] Lustre: server umount lustre-OST0000 complete [ 726.629496] Lustre: server umount lustre-MDT0000 complete [ 729.878661] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 730.777641] Key type lgssc unregistered [ 730.886352] LNet: 60673:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 730.888790] LNetError: 60673:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 730.896253] LNet: Removed LNI 192.168.204.134@tcp [ 731.174627] Key type .llcrypt unregistered [ 731.175723] Key type ._llcrypt unregistered [ 741.498192] Key type ._llcrypt registered [ 741.499746] Key type .llcrypt registered [ 741.537663] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 741.947807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 741.960380] alg: No test for adler32 (adler32-zlib) [ 742.812317] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 742.886568] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 744.463143] Key type lgssc registered [ 744.799591] Lustre: Echo OBD driver; http://www.lustre.org/ [ 747.190868] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 747.196468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 748.339755] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 749.552318] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 751.700750] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 753.812972] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 753.913703] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 755.665554] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 757.765661] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 758.051548] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 758.071378] Lustre: 63480:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 758.074371] Lustre: 63480:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 758.077132] Lustre: 63480:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 758.079499] Lustre: 63480:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 758.081596] Lustre: 63480:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 758.083596] Lustre: 63480:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 765.870531] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 799.200507] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 799.200588] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 799.206686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 804.320736] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 804.429758] Lustre: server umount lustre-OST0000 complete [ 805.722542] Lustre: server umount lustre-MDT0000 complete [ 808.847439] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 809.773038] Key type lgssc unregistered [ 809.886287] LNet: 65062:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 809.888539] LNetError: 65062:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 809.895272] LNet: Removed LNI 192.168.204.134@tcp [ 810.142097] Key type .llcrypt unregistered [ 810.143326] Key type ._llcrypt unregistered [ 815.525630] Key type ._llcrypt registered [ 815.526606] Key type .llcrypt registered [ 815.561239] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 821.297745] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 821.750307] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 821.765762] alg: No test for adler32 (adler32-zlib) [ 822.630852] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 822.716607] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 824.295142] Key type lgssc registered [ 824.638057] Lustre: Echo OBD driver; http://www.lustre.org/ [ 835.909327] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 835.915071] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 837.005400] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 837.016591] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 837.056496] Lustre: lustre-MDT0000: new disk, initializing [ 837.080617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 837.086919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 838.466690] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 842.594808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 842.632304] Lustre: 68187:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 842.656816] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 842.659338] Lustre: Skipped 1 previous similar message [ 842.695216] Lustre: lustre-MDT0001: new disk, initializing [ 842.729751] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 842.745418] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 842.749539] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 844.108707] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 847.926412] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 849.557441] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 851.608698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 851.699180] Lustre: lustre-OST0000: new disk, initializing [ 851.700894] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 851.704099] Lustre: 70156:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 851.742963] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 853.462722] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 856.819368] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 857.063991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 857.066931] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 857.077602] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 858.056924] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 858.122612] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 859.302537] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 859.365345] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 862.178446] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 862.178535] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 862.179619] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 862.180869] LustreError: Skipped 1 previous similar message [ 862.183889] Lustre: Skipped 1 previous similar message [ 866.062359] Lustre: server umount lustre-OST0000 complete [ 867.296242] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 867.296728] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 867.300826] Lustre: Skipped 1 previous similar message [ 867.302657] Lustre: Skipped 1 previous similar message [ 872.415869] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 872.417860] Lustre: Skipped 1 previous similar message [ 873.366666] Lustre: server umount lustre-MDT0000 complete [ 874.508925] LustreError: 68181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684900 with bad export cookie 18275663404572280300 [ 874.511893] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 874.512378] LustreError: 68181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 874.616629] Lustre: server umount lustre-MDT0001 complete [ 877.057085] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:28:23 (1781684903) [ 880.044791] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 892.065672] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 894.933603] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 905.348330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 905.425653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 905.436397] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 905.468469] Lustre: lustre-MDT0000: new disk, initializing [ 905.488650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 905.494286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 906.713976] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 910.552828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 910.591396] Lustre: 76173:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 910.613479] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 910.615184] Lustre: Skipped 1 previous similar message [ 910.649029] Lustre: lustre-MDT0001: new disk, initializing [ 910.679806] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 910.690585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 910.694031] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 911.937944] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 915.302080] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 916.968657] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 919.157228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 919.258604] Lustre: lustre-OST0000: new disk, initializing [ 919.260451] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 919.262962] Lustre: 78138:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 919.287051] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 921.042703] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 924.149625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 924.152523] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 924.177158] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 924.437265] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 925.683119] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 925.742117] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 926.966950] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 927.031863] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 929.250522] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 929.250576] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 929.251475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 929.252613] LustreError: Skipped 1 previous similar message [ 929.261690] Lustre: Skipped 1 previous similar message [ 933.710194] Lustre: server umount lustre-OST0000 complete [ 936.415745] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 936.417579] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 936.420935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 936.422296] Lustre: Skipped 1 previous similar message [ 944.607854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 944.609351] Lustre: Skipped 3 previous similar messages [ 949.215105] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 949.282906] Lustre: server umount lustre-MDT0000 complete [ 950.578133] LustreError: 76163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781684977 with bad export cookie 18275663404572281168 [ 950.582071] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 950.582972] LustreError: 76163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 950.700858] Lustre: server umount lustre-MDT0001 complete [ 953.535352] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 04:29:39 (1781684979) [ 955.658904] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 955.814113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 957.068840] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 959.683612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 961.013488] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 963.210844] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 964.829023] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 966.833960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 968.787698] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 970.961249] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 972.083948] Lustre: 83219:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 09119e89-6acb-4eeb-a747-e27d6bb4805b at adminstrative request [ 973.792097] LustreError: MGC192.168.204.134@tcp: operation ldlm_enqueue to node 0@lo failed: rc = -107 [ 973.794413] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 973.798051] Lustre: 80542:0:(mgc_request.c:1901:mgc_process_log()) MGC192.168.204.134@tcp: IR log lustre-mdtir failed, not fatal: rc = -107 [ 973.798346] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 984.033033] Lustre: MGS: Client 09119e89-6acb-4eeb-a747-e27d6bb4805b (at 0@lo) reconnecting [ 984.036749] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xfda03309b0230cb4 to 0xfda03309b0230f2a [ 984.040176] Lustre: MGC192.168.204.134@tcp: Connection restored to 0@lo (at 0@lo) [ 1004.554424] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1004.630622] Lustre: lustre-OST0001: new disk, initializing [ 1004.632287] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1004.634710] Lustre: 83615:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1004.661312] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1004.664834] Lustre: Skipped 2 previous similar messages [ 1006.569591] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1010.334927] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1011.294926] Lustre: server umount lustre-OST0001 complete [ 1012.215374] LustreError: 82369:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1012.219436] LustreError: 82369:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1016.288477] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1016.290766] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1016.294141] Lustre: Skipped 1 previous similar message [ 1016.295653] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1016.297690] Lustre: Skipped 2 previous similar messages [ 1017.314296] LustreError: 82912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1017.319400] LustreError: 82912:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1018.577124] Lustre: server umount lustre-OST0000 complete [ 1034.207095] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1034.270602] Lustre: server umount lustre-MDT0000 complete [ 1035.536523] LustreError: 82391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685061 with bad export cookie 18275663404572282666 [ 1035.539960] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1035.541171] LustreError: 82391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1035.682726] Lustre: server umount lustre-MDT0001 complete [ 1038.302996] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 04:31:04 (1781685064) [ 1038.799818] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1039.372748] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 04:31:05 (1781685065) [ 1039.884987] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1040.471986] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 04:31:06 (1781685066) [ 1040.965756] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1041.497351] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 04:31:07 (1781685067) [ 1041.986257] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1042.561372] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 04:31:08 (1781685068) [ 1043.095092] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1044.968764] loop: module loaded [ 1044.973422] loop0: detected capacity change from 0 to 4294967296 [ 1044.986394] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1045.458940] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1046.382173] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1047.552324] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1055.685153] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1055.700104] LustreError: 87726:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1055.702183] LustreError: 87726:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1055.704118] LustreError: 87726:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1055.706646] LustreError: 87726:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1061.343406] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 1064.024575] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 1074.889879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1074.981493] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1074.991909] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1075.024677] Lustre: lustre-MDT0000: new disk, initializing [ 1075.047232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1075.054211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1076.277166] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1080.111050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1080.148088] Lustre: 90586:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1080.239354] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1081.623727] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1085.225885] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1087.032517] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1089.181560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1089.280779] Lustre: 92553:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1091.119581] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1094.608846] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1094.647968] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1094.681510] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1095.882042] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1095.960949] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1097.363242] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1097.442824] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1099.745515] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1099.745667] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1099.748570] LustreError: Skipped 3 previous similar messages [ 1099.752145] Lustre: Skipped 3 previous similar messages [ 1099.752786] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1099.758285] Lustre: Skipped 11 previous similar messages [ 1100.178023] Lustre: server umount lustre-OST0000 complete [ 1107.609221] Lustre: server umount lustre-MDT0000 complete [ 1108.795076] LustreError: 90578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685135 with bad export cookie 18275663404572283296 [ 1108.798283] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1108.798590] LustreError: 90578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1108.913203] Lustre: server umount lustre-MDT0001 complete [ 1111.612546] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 04:32:17 (1781685137) [ 1113.813747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1113.972960] LustreError: 94957:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1115.411483] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1118.288196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1119.823455] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1122.044432] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1123.726895] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1125.741092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1127.776678] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1130.123206] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1131.610524] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1140.842508] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 9 sec [ 1142.268276] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1142.345547] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1144.561325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1144.626420] Lustre: lustre-OST0001: new disk, initializing [ 1144.628334] Lustre: Skipped 2 previous similar messages [ 1144.630288] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1144.631954] Lustre: Skipped 3 previous similar messages [ 1144.634799] Lustre: 98391:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1146.548951] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1150.126351] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1151.464297] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1153.535745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1153.537987] Lustre: Skipped 2 previous similar messages [ 1153.539636] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1153.566410] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1154.589135] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1155.791536] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1155.857471] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1165.670960] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1170.194456] Lustre: Failing over lustre-MDT0000 [ 1170.521721] Lustre: server umount lustre-MDT0000 complete [ 1172.563592] LustreError: 94953:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1172.569030] LustreError: 94953:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1173.987582] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1173.991712] Lustre: Skipped 6 previous similar messages [ 1177.676046] LustreError: 97689:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1177.683211] LustreError: 97689:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1185.106893] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1185.109167] LDISKFS-fs (dm-0): recovery complete [ 1185.112847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1185.156841] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1185.287757] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1186.598123] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1187.916195] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1187.918094] Lustre: 96437:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1190.370751] Lustre: 94958:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1190.374231] Lustre: 94958:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 1190.378533] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1190.395902] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1190.711054] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1191.025874] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1191.029733] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 1191.343111] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1191.346697] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 1191.665983] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1191.672926] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 8 previous similar messages [ 1192.305585] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1192.308542] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 1192.623128] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1192.626352] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 1193.880199] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1193.885241] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 1194.512114] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1194.516523] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 1194.831134] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1194.833902] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 1197.961679] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1197.966629] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 1198.593211] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1198.597657] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 1198.911123] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1198.913880] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 1206.097575] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1206.101517] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 25 previous similar messages [ 1206.729566] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1206.733201] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 1207.047128] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1207.049475] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 1222.345065] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1222.350038] Lustre: 101230:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 51 previous similar messages [ 1222.977459] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1222.981131] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 51 previous similar messages [ 1223.295078] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1223.297353] LustreError: 101230:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 51 previous similar messages [ 1247.071400] Lustre: 101230:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1247.075606] Lustre: 101230:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1247.079776] Lustre: 101230:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d3ee4e58-c625-4cde-8ec2-2ba196e24053@192.168.204.34@tcp [ 1247.086838] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1247.091646] Lustre: 101230:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@00000000fe0d6a37 x1868231966001408/t0(8589934778) o35->d3ee4e58-c625-4cde-8ec2-2ba196e24053@192.168.204.34@tcp:44/0 lens 392/0 e 0 to 0 dl 1781685284 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1247.108949] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1247.117606] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1247.132211] Lustre: lustre-MDT0000: Denying connection for new client d3ee4e58-c625-4cde-8ec2-2ba196e24053 (at 192.168.204.34@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1247.147428] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1247.149201] Lustre: Skipped 2 previous similar messages [ 1247.152916] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1247.176915] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 1247.177344] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 1257.953855] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1257.960119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1258.091147] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1258.094584] Lustre: Skipped 2 previous similar messages [ 1263.561329] Lustre: server umount lustre-OST0000 complete [ 1267.174429] LustreError: 101609:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1267.190095] LustreError: 101609:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 1271.862835] Lustre: server umount lustre-OST0001 complete [ 1280.533056] Lustre: server umount lustre-MDT0000 complete [ 1284.610274] LustreError: 98390:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685311 with bad export cookie 18275663404572334480 [ 1284.625253] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1285.040940] Lustre: server umount lustre-MDT0001 complete [ 1291.323410] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 04:35:16 (1781685316) [ 1295.762742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1295.768309] Lustre: *** cfs_fail_loc=197, val=0*** [ 1311.263607] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1316.383566] LustreError: 103204:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.134@tcp: failed processing log, type 4: rc = -110 [ 1321.511078] Lustre: *** cfs_fail_loc=197, val=0*** [ 1321.512802] Lustre: Skipped 65 previous similar messages [ 1341.984367] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1341.987149] Lustre: Skipped 7 previous similar messages [ 1344.824365] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1347.906590] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1348.827923] Lustre: server umount lustre-OST0000 complete [ 1352.651200] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 04:36:18 (1781685378) [ 1366.864034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1367.038319] LustreError: 104822:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1367.047589] LustreError: 104822:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1368.530478] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1372.753209] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1375.316202] LustreError: 104854:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1375.324244] LustreError: 104854:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1375.328390] LustreError: 104854:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -5 [ 1375.518738] Lustre: server umount lustre-MDT0000 complete [ 1390.000896] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 1394.141090] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 1410.389169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1410.520668] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1410.542372] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1410.590778] Lustre: lustre-MDT0000: new disk, initializing [ 1410.661902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1412.621379] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1418.716205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1418.794411] Lustre: 109019:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1418.998856] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1421.133099] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1425.982729] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1428.587073] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1431.870393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1431.998648] Lustre: 110985:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1434.811579] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1439.600310] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1441.336070] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1441.502702] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1441.596290] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1443.475229] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1443.571255] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1446.370468] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1446.370822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1446.373180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1446.373186] Lustre: Skipped 8 previous similar messages [ 1446.374304] LustreError: Skipped 3 previous similar messages [ 1446.384886] Lustre: Skipped 6 previous similar messages [ 1450.587966] Lustre: server umount lustre-OST0000 complete [ 1458.409342] Lustre: server umount lustre-MDT0000 complete [ 1460.078433] LustreError: 109811:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685486 with bad export cookie 18275663404572345162 [ 1460.080592] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1460.086083] LustreError: 109811:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1460.257927] Lustre: server umount lustre-MDT0001 complete [ 1463.810453] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 04:38:09 (1781685489) [ 1467.191532] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 1468.408840] Key type lgssc unregistered [ 1468.551903] LNet: 113466:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1468.555334] LNetError: 113466:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1468.562418] LNet: Removed LNI 192.168.204.134@tcp [ 1468.918929] Key type .llcrypt unregistered [ 1468.920643] Key type ._llcrypt unregistered [ 1476.220503] Key type ._llcrypt registered [ 1476.222419] Key type .llcrypt registered [ 1476.265961] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 1483.748869] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 1484.130453] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1484.180505] alg: No test for adler32 (adler32-zlib) [ 1485.057305] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 1485.148845] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 1486.743171] Key type lgssc registered [ 1487.214334] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1502.187344] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1502.195418] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1503.291554] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1503.303242] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1503.341085] Lustre: lustre-MDT0000: new disk, initializing [ 1503.372597] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1503.382273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1504.807255] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1509.086119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1509.198186] Lustre: lustre-OST0000: new disk, initializing [ 1509.200576] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1509.202992] Lustre: Skipped 1 previous similar message [ 1509.205589] Lustre: 117611:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1509.235112] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1511.242513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1511.247619] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1511.261188] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1511.403728] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1516.068108] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1516.165366] Lustre: lustre-OST0001: new disk, initializing [ 1516.169161] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1516.173834] Lustre: 118481:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1516.214191] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1517.583173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1517.587973] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1517.601033] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1518.595523] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1527.777484] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1527.780117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1527.789493] Lustre: Skipped 1 previous similar message [ 1529.617231] Lustre: server umount lustre-MDT0000 complete [ 1532.719442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1534.802506] LustreError: 116799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685561 with bad export cookie 15647484207157337130 [ 1534.805149] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1534.809553] LustreError: 116799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1534.851942] Lustre: server umount lustre-OST0000 complete [ 1536.683119] Lustre: server umount lustre-OST0001 complete [ 1540.668758] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 04:39:26 (1781685566) [ 1544.857309] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 1555.093823] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 04:39:40 (1781685580) [ 1572.591288] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 1576.762332] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 1592.628545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1592.748272] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1592.773086] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1592.824047] Lustre: lustre-MDT0000: new disk, initializing [ 1592.868629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1592.881870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1594.526708] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1599.883232] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1599.940706] Lustre: 125214:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1599.970611] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1599.973500] Lustre: Skipped 1 previous similar message [ 1600.016068] Lustre: lustre-MDT0001: new disk, initializing [ 1600.064844] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1600.085993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1600.094086] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1601.962707] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1606.948869] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1609.471216] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1612.734506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1612.879885] Lustre: lustre-OST0000: new disk, initializing [ 1612.882274] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1612.886294] Lustre: 127182:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1612.927707] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1615.743659] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1616.146176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1616.152203] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1616.201057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1620.748550] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1622.946569] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1623.060561] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1625.254926] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1625.384247] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1626.597817] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1626.597990] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1626.600612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1626.600620] Lustre: Skipped 1 previous similar message [ 1626.603413] LustreError: Skipped 1 previous similar message [ 1626.610920] Lustre: Skipped 1 previous similar message [ 1631.717047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1631.722863] Lustre: Skipped 2 previous similar messages [ 1632.552726] Lustre: server umount lustre-OST0000 complete [ 1635.808056] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1635.811938] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1635.817709] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1640.365280] Lustre: server umount lustre-MDT0000 complete [ 1642.112136] LustreError: 126010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685668 with bad export cookie 15647484207157338348 [ 1642.112615] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1642.121385] LustreError: 126010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1642.295567] Lustre: server umount lustre-MDT0001 complete [ 1646.228595] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 04:41:12 (1781685672) [ 1649.169518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1649.374143] LustreError: 129591:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1649.384873] LustreError: 129591:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1649.407801] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1651.149668] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1654.752659] LustreError: 129592:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1654.909275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1657.073407] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1660.496256] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1662.775521] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1665.709412] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1668.344396] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1671.821171] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1683.488942] Lustre: *** cfs_fail_loc=159, val=0*** [ 1699.948292] Lustre: lustre-MDT0000: Client 8ac38964-7ab4-45d8-8fc2-70540131b9cb (at 192.168.204.34@tcp) reconnecting [ 1700.703267] Lustre: *** cfs_fail_loc=159, val=0*** [ 1700.705622] Lustre: Skipped 5 previous similar messages [ 1717.328493] Lustre: lustre-MDT0000: Client 8ac38964-7ab4-45d8-8fc2-70540131b9cb (at 192.168.204.34@tcp) reconnecting [ 1735.647944] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1735.650596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1735.655163] Lustre: Skipped 1 previous similar message [ 1735.657554] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1735.659983] Lustre: Skipped 2 previous similar messages [ 1738.524715] Lustre: server umount lustre-OST0000 complete [ 1741.793897] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1741.798222] Lustre: Skipped 2 previous similar messages [ 1746.013687] Lustre: server umount lustre-MDT0000 complete [ 1746.913541] LustreError: 129592:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1746.921516] LustreError: 129592:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1747.518525] LustreError: 129572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685773 with bad export cookie 15647484207157339230 [ 1747.523627] LustreError: 129572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1747.523861] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1747.658715] Lustre: server umount lustre-MDT0001 complete [ 1751.710957] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 1752.867141] Key type lgssc unregistered [ 1753.025536] LNet: 133385:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1753.029057] LNetError: 133385:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1753.038445] LNet: Removed LNI 192.168.204.134@tcp [ 1753.353856] Key type .llcrypt unregistered [ 1753.355247] Key type ._llcrypt unregistered [ 1771.448011] hrtimer: interrupt took 3013602 ns [ 1771.571782] Key type ._llcrypt registered [ 1771.573335] Key type .llcrypt registered [ 1771.671258] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 1772.918486] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1772.978865] alg: No test for adler32 (adler32-zlib) [ 1774.650098] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 1774.824825] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 1776.535167] Key type lgssc registered [ 1777.990812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1784.996331] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1785.021658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1786.446734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1790.899835] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1796.582225] LustreError: 134795:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1796.602199] LustreError: 134795:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1799.638784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1799.904748] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1804.109578] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1810.479340] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1815.657563] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1823.108257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1823.575970] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1829.457433] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 1831.791268] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1835.966864] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1844.322978] Lustre: *** cfs_fail_loc=159, val=0*** [ 1859.690348] Lustre: lustre-MDT0000: Client 29112c88-8bec-4c2f-b2a1-e7f8f8245344 (at 192.168.204.34@tcp) reconnecting [ 1861.499597] Lustre: *** cfs_fail_loc=159, val=0*** [ 1878.137499] Lustre: lustre-MDT0001: Client 29112c88-8bec-4c2f-b2a1-e7f8f8245344 (at 192.168.204.34@tcp) reconnecting [ 1879.111675] Lustre: *** cfs_fail_loc=159, val=0*** [ 1879.119459] Lustre: Skipped 3 previous similar messages [ 1894.499717] Lustre: lustre-MDT0001: Client 29112c88-8bec-4c2f-b2a1-e7f8f8245344 (at 192.168.204.34@tcp) reconnecting [ 1898.533100] Lustre: 137831:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1901.695192] Lustre: *** cfs_fail_loc=159, val=0*** [ 1901.706310] Lustre: Skipped 2 previous similar messages [ 1917.762629] Lustre: lustre-MDT0000: Client b943e62a-d6c3-4c29-a081-6f75701abf5a (at 192.168.204.34@tcp) reconnecting [ 1920.532854] Lustre: *** cfs_fail_loc=159, val=0*** [ 1920.540718] Lustre: Skipped 14 previous similar messages [ 1937.026196] Lustre: lustre-MDT0000: Client b943e62a-d6c3-4c29-a081-6f75701abf5a (at 192.168.204.34@tcp) reconnecting [ 1938.081442] Lustre: 138122:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1942.496638] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1942.504643] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1942.513618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1943.011839] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1943.015113] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1943.018972] LustreError: Skipped 2 previous similar messages [ 1943.039677] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1943.771702] Lustre: server umount lustre-OST0000 complete [ 1948.639931] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1948.646971] LustreError: Skipped 3 previous similar messages [ 1948.650666] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1948.663996] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1952.567609] Lustre: server umount lustre-MDT0000 complete [ 1954.784696] LustreError: 137549:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1954.799162] LustreError: 137549:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1954.962721] LustreError: 134778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781685981 with bad export cookie 11760442789358872183 [ 1954.963988] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1954.971372] LustreError: 134778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1955.219433] Lustre: server umount lustre-MDT0001 complete [ 1963.519656] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 1965.850523] Key type lgssc unregistered [ 1966.087805] LNet: 139173:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1966.093279] LNetError: 139173:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1966.111663] LNet: Removed LNI 192.168.204.134@tcp [ 1966.878391] Key type .llcrypt unregistered [ 1966.885541] Key type ._llcrypt unregistered [ 1986.721053] Key type ._llcrypt registered [ 1986.722831] Key type .llcrypt registered [ 1986.818714] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 1987.844356] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1987.857569] alg: No test for adler32 (adler32-zlib) [ 1988.859394] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 1989.053849] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 1990.695236] Key type lgssc registered [ 1991.784633] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1998.556915] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1998.577494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2000.050858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2003.675413] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2009.571668] LustreError: 140587:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2009.584460] LustreError: 140587:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2010.769263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2011.136421] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2015.026251] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2021.239594] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2025.774394] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2031.100077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2031.352872] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2036.317444] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2036.516819] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2041.884358] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2044.664810] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2050.648487] Lustre: 143168:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2052.625615] Lustre: 143218:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2053.513229] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2054.089525] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 2056.373221] Lustre: 143364:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2057.035378] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2057.613646] Lustre: 143462:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2060.258645] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2060.258799] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2060.266642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2060.273509] Lustre: Skipped 1 previous similar message [ 2065.388519] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2065.392642] Lustre: Skipped 2 previous similar messages [ 2066.325559] Lustre: server umount lustre-OST0000 complete [ 2070.512759] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2070.530697] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2070.535965] Lustre: Skipped 1 previous similar message [ 2075.619813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2075.625527] Lustre: Skipped 2 previous similar messages [ 2076.373642] Lustre: server umount lustre-MDT0000 complete [ 2079.966874] LustreError: 140568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781686106 with bad export cookie 14649785680982932034 [ 2079.969765] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2079.986323] LustreError: 140568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2080.291294] Lustre: server umount lustre-MDT0001 complete [ 2091.526817] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 2093.765268] Key type lgssc unregistered [ 2093.970923] LNet: 144519:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2093.976665] LNetError: 144519:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2093.992922] LNet: Removed LNI 192.168.204.134@tcp [ 2094.612567] Key type .llcrypt unregistered [ 2094.616832] Key type ._llcrypt unregistered [ 2116.380557] Key type ._llcrypt registered [ 2116.381792] Key type .llcrypt registered [ 2116.446294] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 2117.198571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2117.280307] alg: No test for adler32 (adler32-zlib) [ 2118.301173] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 2118.500906] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 2120.183247] Key type lgssc registered [ 2121.061863] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2127.331891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2127.353741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2128.814858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2132.354298] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2139.108403] LustreError: 145936:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2139.120766] LustreError: 145936:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2139.804727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2140.129181] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2143.608447] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2149.091263] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2153.539439] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2158.748555] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2159.086429] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2164.728245] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2165.230044] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2171.553829] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2174.460478] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2180.556566] Lustre: *** cfs_fail_loc=159, val=0*** [ 2197.110733] Lustre: lustre-MDT0000: Client 67c54b49-0718-4917-bb39-cd7eb59f8240 (at 192.168.204.34@tcp) reconnecting [ 2201.056253] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2201.070924] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2201.086299] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2203.111057] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 2203.114962] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2203.121540] LustreError: Skipped 1 previous similar message [ 2203.139981] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2205.152718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2205.156535] Lustre: Skipped 1 previous similar message [ 2206.526623] Lustre: server umount lustre-OST0000 complete [ 2210.273637] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2210.276530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2210.284523] Lustre: Skipped 1 previous similar message [ 2210.297100] Lustre: Skipped 1 previous similar message [ 2215.392746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2217.738617] Lustre: server umount lustre-MDT0000 complete [ 2220.517285] LustreError: 145935:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2220.530915] LustreError: 145935:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2220.545690] LustreError: 145916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781686246 with bad export cookie 7886149529343786259 [ 2220.548782] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2220.559265] LustreError: 145916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2220.808656] Lustre: server umount lustre-MDT0001 complete [ 2229.329875] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 2231.506222] Key type lgssc unregistered [ 2231.755468] LNet: 149598:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2231.760228] LNetError: 149598:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2231.772482] LNet: Removed LNI 192.168.204.134@tcp [ 2232.448387] Key type .llcrypt unregistered [ 2232.451931] Key type ._llcrypt unregistered [ 2254.723241] Key type ._llcrypt registered [ 2254.724719] Key type .llcrypt registered [ 2254.826467] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 2255.898802] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2255.921674] alg: No test for adler32 (adler32-zlib) [ 2256.910082] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 2257.113300] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 2258.775174] Key type lgssc registered [ 2259.713503] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2266.026158] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2266.044716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2267.484140] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2271.033872] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2276.841115] LustreError: 151012:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2276.856519] LustreError: 151012:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2278.129842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2278.545210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2281.978345] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2287.878752] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2292.581931] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2298.123047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2298.448370] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2303.205951] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2303.598603] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2309.597784] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2311.811644] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2330.872330] Lustre: lustre-MDT0001: evicting f27c52a2-4802-4416-bded-df55a67436f7 (at 192.168.204.34@tcp) by administrative request [ 2330.880760] Lustre: Skipped 1 previous similar message [ 2358.243527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2358.255343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2358.259726] Lustre: Skipped 1 previous similar message [ 2358.267858] Lustre: Skipped 1 previous similar message [ 2363.361421] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2363.372323] Lustre: Skipped 1 previous similar message [ 2364.111599] Lustre: server umount lustre-OST0000 complete [ 2368.483476] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2368.499843] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2368.509411] Lustre: Skipped 1 previous similar message [ 2373.504256] Lustre: server umount lustre-MDT0000 complete [ 2373.609806] LustreError: 151008:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2373.625231] LustreError: 151008:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2376.569041] LustreError: 150993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781686403 with bad export cookie 14441222186486937643 [ 2376.572854] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2376.581612] LustreError: 150993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2376.887325] Lustre: server umount lustre-MDT0001 complete [ 2385.546659] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 2388.090960] Key type lgssc unregistered [ 2388.352705] LNet: 155080:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2388.359614] LNetError: 155080:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2388.390795] LNet: Removed LNI 192.168.204.134@tcp [ 2389.214865] Key type .llcrypt unregistered [ 2389.216450] Key type ._llcrypt unregistered [ 2446.333926] Key type ._llcrypt registered [ 2446.338507] Key type .llcrypt registered [ 2446.433602] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 2447.372444] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2447.409216] alg: No test for adler32 (adler32-zlib) [ 2448.417402] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 2448.575223] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 2450.215298] Key type lgssc registered [ 2451.062744] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2457.289994] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2457.312734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2458.705916] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2462.495625] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2468.686231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2468.835699] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 2468.934166] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2472.231856] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2478.320202] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2483.013037] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2487.705815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2487.956591] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2492.671741] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2494.137368] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2498.195907] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2504.380159] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2520.039986] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2520.042849] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2520.056673] LustreError: Skipped 1 previous similar message [ 2520.078301] Lustre: Skipped 1 previous similar message [ 2520.082655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2521.194579] Lustre: server umount lustre-OST0000 complete [ 2525.156989] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2525.162783] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2525.169167] Lustre: Skipped 1 previous similar message [ 2525.182099] Lustre: Skipped 2 previous similar messages [ 2530.281997] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2530.295968] Lustre: Skipped 1 previous similar message [ 2530.868816] Lustre: server umount lustre-MDT0000 complete [ 2533.740607] LustreError: 158286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781686560 with bad export cookie 10522105470709022440 [ 2533.745511] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2533.750365] LustreError: 158286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2534.010864] Lustre: server umount lustre-MDT0001 complete [ 2542.310536] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 2544.475617] Key type lgssc unregistered [ 2544.722525] LNet: 161112:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2544.728556] LNetError: 161112:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2544.740230] LNet: Removed LNI 192.168.204.134@tcp [ 2545.352954] Key type .llcrypt unregistered [ 2545.358258] Key type ._llcrypt unregistered [ 2564.103547] Key type ._llcrypt registered [ 2564.106647] Key type .llcrypt registered [ 2564.209911] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 2565.001099] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2565.052544] alg: No test for adler32 (adler32-zlib) [ 2566.083409] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 2566.292862] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 2567.952340] Key type lgssc registered [ 2568.896067] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2579.965439] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 04:56:45 (1781686605) [ 2591.371558] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 2598.207953] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 2621.615464] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2621.633557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2622.815967] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2622.835280] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2622.892226] Lustre: lustre-MDT0000: new disk, initializing [ 2622.987621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2623.002872] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2625.552665] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2632.914099] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2632.989055] Lustre: 165353:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2633.048098] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2633.054065] Lustre: Skipped 1 previous similar message [ 2633.116745] Lustre: lustre-MDT0001: new disk, initializing [ 2633.196564] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2633.226267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2633.230410] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2636.515386] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2643.643404] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2647.755274] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2652.676647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2652.892152] Lustre: lustre-OST0000: new disk, initializing [ 2652.895181] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2652.901122] Lustre: 167317:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2652.967226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2657.381731] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2658.868500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2658.874563] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2658.955269] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2664.353697] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2673.617690] Lustre: server umount lustre-OST0000 complete [ 2679.266901] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2679.275298] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2679.284578] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2679.777100] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2679.791238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2684.896507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2684.902313] Lustre: Skipped 1 previous similar message [ 2685.990897] Lustre: server umount lustre-MDT0000 complete [ 2688.564677] LustreError: 167316:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781686715 with bad export cookie 7715169470292073992 [ 2688.566084] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2688.572837] LustreError: 167316:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2698.851758] Lustre: server umount lustre-MDT0001 complete [ 2705.962170] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 2707.822037] Key type lgssc unregistered [ 2708.047222] LNet: 169225:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2708.053156] LNetError: 169225:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2708.068603] LNet: Removed LNI 192.168.204.134@tcp [ 2708.644501] Key type .llcrypt unregistered [ 2708.646581] Key type ._llcrypt unregistered [ 2724.978521] Key type ._llcrypt registered [ 2724.980684] Key type .llcrypt registered [ 2725.056960] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 2725.612261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2725.637068] alg: No test for adler32 (adler32-zlib) [ 2726.648991] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 2726.807141] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 2728.511399] Key type lgssc registered [ 2729.350772] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2734.332699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2734.348935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2735.633990] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2738.225368] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2743.951250] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2744.189588] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2746.851210] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2751.893580] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2755.599657] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2760.358380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2760.639939] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2761.652060] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2764.600839] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 2769.309107] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2773.632750] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2773.651517] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2773.662200] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2773.671324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.683629] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2774.263840] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2774.277742] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2774.286882] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2774.291036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.298546] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2774.883258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.455900] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2775.460891] Lustre: Skipped 1 previous similar message [ 2775.471162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2775.477485] Lustre: Skipped 1 previous similar message [ 2775.480206] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2775.483494] Lustre: Skipped 1 previous similar message [ 2775.485827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.494020] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2775.499726] Lustre: Skipped 1 previous similar message [ 2776.070702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.635410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.235966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.794484] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2777.799955] Lustre: Skipped 3 previous similar messages [ 2777.811182] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2777.817252] Lustre: Skipped 3 previous similar messages [ 2777.820573] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2777.827023] Lustre: Skipped 3 previous similar messages [ 2777.830582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.838918] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2777.843779] Lustre: Skipped 3 previous similar messages [ 2778.340674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.910322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.578267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.146278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.723644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.317191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.857822] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2781.861769] Lustre: Skipped 6 previous similar messages [ 2781.876609] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2781.884881] Lustre: Skipped 6 previous similar messages [ 2781.888167] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2781.891185] Lustre: Skipped 6 previous similar messages [ 2781.895119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.901706] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2781.906244] Lustre: Skipped 6 previous similar messages [ 2782.500762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.144253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.688077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.373674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.888236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.470181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.035177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.651612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.319832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.950926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.543708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.215625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.801166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.438735] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2790.442798] Lustre: Skipped 13 previous similar messages [ 2790.456324] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2790.463570] Lustre: Skipped 13 previous similar messages [ 2790.468693] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2790.472795] Lustre: Skipped 13 previous similar messages [ 2790.477764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.489238] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2790.495094] Lustre: Skipped 13 previous similar messages [ 2791.067798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.680435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.253400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.920270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.534187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.168975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.758770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.373285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.957758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.621641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.256319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.935966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.548817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.245931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.809503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.428623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.134069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.777447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.440436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.054378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.693429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.339906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.949671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.568526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.187254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.778372] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2806.785853] Lustre: Skipped 25 previous similar messages [ 2806.799157] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2806.807314] Lustre: Skipped 25 previous similar messages [ 2806.811307] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2806.816522] Lustre: Skipped 25 previous similar messages [ 2806.819173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.834452] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2806.838073] Lustre: Skipped 25 previous similar messages [ 2807.429093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.124812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.709456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.309834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.884708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.494970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.149069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.726295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.338853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.996111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.607290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.238367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.844698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.420147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.958484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.554640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.116486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.697722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.310361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.922479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.471892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.108696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.681176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.266230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.859208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.467743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.131926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.803425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.483875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.133498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.825456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.553605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.249712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.962407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.663662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.257695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.953587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.556189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.177158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.818365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.422381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.050184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.715559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.299231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.929163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.690760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.529751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.216274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.896763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.469380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.167931] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2839.174548] Lustre: Skipped 50 previous similar messages [ 2839.185033] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2839.192207] Lustre: Skipped 50 previous similar messages [ 2839.197232] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2839.204153] Lustre: Skipped 50 previous similar messages [ 2839.208958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.219680] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2839.223271] Lustre: Skipped 50 previous similar messages [ 2839.852360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.513062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.190580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.852976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.449539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.065079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.686611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.327381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.940531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.503307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.078146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.673185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.265648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.825628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.441736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.082102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.681069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.270042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.851169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.460990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.018415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.591439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.182680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.818400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.431897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.006929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.581961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.246279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.850977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.416246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.977104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.614080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.304683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.972129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.532245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.126074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.665593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.181718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.703101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.238830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.739957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.296942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.872064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.463531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.035478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.638610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.246087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.884344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.526719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.122812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.709304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.273251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.895761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.519454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.118909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.675151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.277377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.804346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.329499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.836752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.371476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.882573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.428049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.007239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.580799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.139246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.708552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.282301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.900421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.464708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.051722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.609395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.181556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.752535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.377055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.949816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.481659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.010959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.544144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.123958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.727625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.294969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.893551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.440972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.039065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.576727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.208879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.767433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.341549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.902183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.491461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.035856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.604944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.224363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.834317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.433682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.936861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.488263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.009618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.664298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.208648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.815876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.319376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.812421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.422450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.925962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.465504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.988584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.520364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.054313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.619864] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2903.623691] Lustre: Skipped 110 previous similar messages [ 2903.634610] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2903.640294] Lustre: Skipped 110 previous similar messages [ 2903.642370] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2903.644795] Lustre: Skipped 110 previous similar messages [ 2903.650377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.656678] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2903.660539] Lustre: Skipped 110 previous similar messages [ 2904.181563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.714220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.212800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.743920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.292967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.832404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.411250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.928339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.380188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.868219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.375477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.869201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.384694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.985825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.512500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.020649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.588401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.185214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.791805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.396081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.973829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.546808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.128745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.743409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.335574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.009942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.651976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.280948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.929887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.582509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.189492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.787045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.428965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.077462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.740866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.422560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.991508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.620954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.225217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.972324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.631943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.285383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.899448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.606433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.165314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.754511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.275072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.813892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.312841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.815061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.347626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.808541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.316792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.838542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.354730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.855764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.415511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.948205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.491517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.025682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.572555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.143312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.674967] LustreError: 170654:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2939.679731] LustreError: 170654:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2939.684454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.246830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.801220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.363147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.980957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.602703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.174883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.842204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.489944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.086114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.697580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.331561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.984114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.614733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.251101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.825321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.366659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.879857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.408341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.937616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.472422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.072512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.657935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.171733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.682666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.258719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.893516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.448470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.020537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.597682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.156028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.764347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.317686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.892562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.433185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.961082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.494224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.041300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.644373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.196227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.688817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.242498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.744123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.293716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.864525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.473132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.028622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.685754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.269524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.787762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.305775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.845218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.342566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.811901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.313740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.866765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.395959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.936391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.526287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.067731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.585308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.111722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.635688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.182967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.755725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.258261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.835308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.360946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.873809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.448579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.937366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.425038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.960212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.490552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.035271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.641648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.237734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.752313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.301497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.863751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.437325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.051369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.740610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.296853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.915659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.514948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.091668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.639451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.221425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.765885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.348419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.958183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.518025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.145794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.723905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.249751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.821687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.349921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.942047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.545920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.119601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.718779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.303927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.847548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.400282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.958551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.574544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.155266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.726517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.245649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.836288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.400973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.945258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.457874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.033604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.567916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.056555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.558266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.099904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.575695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.073752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.560588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.014874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.472733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.912868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.342608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.813253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.276530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.743244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.289722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.765216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.285757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.801919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.300254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.795592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.368308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.880799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.443250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.985107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.594171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.139933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.726052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.218141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.769562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.332139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.873433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.457853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.023742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.573091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.130560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.709463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.356265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.866968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.405151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.966846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.477291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.991729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.525333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.999401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.508797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.084961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.615242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.094777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.628772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.110793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.627615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.164787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.656983] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3031.659858] Lustre: Skipped 229 previous similar messages [ 3031.668951] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3031.676711] Lustre: Skipped 229 previous similar messages [ 3031.680115] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3031.683308] Lustre: Skipped 229 previous similar messages [ 3031.686639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.691608] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3031.697821] Lustre: Skipped 229 previous similar messages [ 3032.181366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.687642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.205049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.821332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.420371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.949583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.441709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.941663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.477211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.981358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.482447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.992377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.467118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.927663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.444628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.929234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.403909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.889479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.372157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.872418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.347095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.868318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.362528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.857635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.372891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.897828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.391668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.846985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.298053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.778825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.256195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.721329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.178910] LustreError: 170654:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3048.187769] LustreError: 170654:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3048.190295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.672151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.184182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.754797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.247247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.720433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.208881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.646327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.087455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.519120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.982362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.427482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.907230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.373020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.828657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.269398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.753555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.218422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.719045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.216438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.689602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.158135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.652536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.167187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.688243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.155572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.630364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.115915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.562146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.017777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.493407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.950579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.364071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.812565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.234295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.694527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.155690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.591840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.008521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.520038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.064181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.624307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.056868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.523289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.026129] LustreError: 170654:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3069.031112] LustreError: 170654:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3069.032829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.453282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.940246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.387323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.845436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.287575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.735056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.151586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.606293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.094184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.554733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.954742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.394902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.830352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.236678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.724056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.193846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.673213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.152233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.628363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.194292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.798729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.367152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.897809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.456326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.972550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.468472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.987445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.446219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.949731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.416251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.927698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.399804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.888934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.404088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.908318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.471308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.993641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.532818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.089836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.549091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.034679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.504031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.956804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.417960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.959532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.503721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.074951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.620288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.151911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.688350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.273744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.840018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.437228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.020549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.542753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.050397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.567148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.153922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.650634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.113474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.590825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.106627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.620935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.129594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.606882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.115387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.611835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.095073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.768250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.359457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.864386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.315968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.740913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.297644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.812291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.310973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.852934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.392105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.988711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.500238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.064491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.574097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.093372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.667795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.251806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.856970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.473744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.029052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.573282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.150077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.685084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.307848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.840756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.412579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.951222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.503549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.108558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.707655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.244855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.863744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.465168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.033944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.720211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.237247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.779507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.415187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.990555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.635175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.270297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.844203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.341220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.881295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.411588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.915116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.468287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.983460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.547130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.052698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.549200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.064590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.581210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.123814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.586234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.160235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.696159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.228461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.814178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.438237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.020554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.536096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.071362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.657619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.212460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.807204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.367072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.910273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.514893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.047678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.516462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.954831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.432484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.912180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.350857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.846688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.258625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.727213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.221831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.695548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.161888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.747707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.259423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.726772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.179941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.639240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.118203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.604347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.104420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.632850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.182563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.685391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.167685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.684473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.165527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.713307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.217701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.702628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.243108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.756578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.232074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.684334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.206156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.665747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.181267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.714453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.271217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.780905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.320771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.868874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.396154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.908591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.455697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.925574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.492226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.034425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.527498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.080321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.634932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.111228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.592331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.077953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.539974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.014971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.509224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.994484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.484884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.979901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.460872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.941582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.492438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.956079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.418231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.908570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.376394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.840028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.339358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.782815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.297703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.807954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.310137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.813977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.293920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.744489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.238445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.737536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.179324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.643359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.118352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.629155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.093661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.606022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.117121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.600831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.146302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.637049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.138869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.647081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.228551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.728516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.218048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.712505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.246152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.798213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.431790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.015110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.606911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.098183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.677853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.168391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.753430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.271370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.717980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.300573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.910558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.429531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.994903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.536333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.152490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.658662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.153796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.674841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.207640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.851427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.460672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.031955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.633920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.201402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.773728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.276726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.824240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.373582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.972528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.478725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.005534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.553735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.030646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.485781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.041642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.522256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.026491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.562059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.049964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.571990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.128028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.614247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.134185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.589652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.130647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.633345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.161860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.716407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.235709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.744713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.247919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.783856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.288503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.784229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.351699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.844053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.315327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.825547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.326780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.799625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.330321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.816722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.327304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.796126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.302314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.731930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.227325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.674286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.157644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.655038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.148316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.606960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.150375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.652766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.163158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.699604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.261889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.770090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.312668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.831616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.366399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.891476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.409828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.927878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.430098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.942561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.404129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.879690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.318101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.770896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.265420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.732857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.224842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.730896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.245206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.813419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.295581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.799164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.345108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.848164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.421675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.951596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.461919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.994363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.594306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.148915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.717480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.244723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.861396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.440447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.041807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.580334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.067094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.584316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.065892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.547171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.001445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.469239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.989215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.522972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.049486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.562406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.058429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.500480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.968665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.482585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.967854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.471242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.960106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.504880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.978951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.409279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.891174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.335576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.832474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.328804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.828794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.333822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.873413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.387172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.878197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.436978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.889600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.410928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.001811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.535261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.012248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.559714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.060229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.597117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.096630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.591274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.143036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.664588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.226308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.756410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.281432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.792280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.313531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.911835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.547160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.136043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.679906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.256906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.774955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.290906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.826870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.338647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.829453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.339773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.856690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.309599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.879190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.371248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.891985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.379613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.923788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.462381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.963733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.457092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.930712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.418037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.941864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.498339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.115565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.625065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.151523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.696345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.179061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.695132] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3287.699723] Lustre: Skipped 498 previous similar messages [ 3287.708188] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3287.714928] Lustre: Skipped 498 previous similar messages [ 3287.717709] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3287.721635] Lustre: Skipped 498 previous similar messages [ 3287.723995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.730704] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3287.736210] Lustre: Skipped 498 previous similar messages [ 3288.223356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.699820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.223269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.744268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.303427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.818104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.317501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.858784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.353914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.844499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.317096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.793599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.335534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.807419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.348827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.855348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.350425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.828586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.335449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.844878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.331263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.823822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.316223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.836230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.380309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.886551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.407729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.944452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.473701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.949121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.405793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.927733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.447359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.004516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.496629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.990690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.533787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.095347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.623638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.141337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.669303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.260235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.781571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.333725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.897372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.385708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.855565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.346364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.825990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.264541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.677447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.176681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.665538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.158250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.797596] LustreError: 172453:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3547.803969] LustreError: 170654:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3698.656920] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3698.661798] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3700.420689] Lustre: server umount lustre-OST0000 complete [ 3702.755584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3702.758701] Lustre: Skipped 1 previous similar message [ 3707.872530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3707.875842] Lustre: Skipped 1 previous similar message [ 3708.132926] Lustre: server umount lustre-MDT0000 complete [ 3709.592653] LustreError: 224688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781687736 with bad export cookie 13873345937355719809 [ 3709.596465] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3709.597561] LustreError: 224688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3720.221493] Lustre: server umount lustre-MDT0001 complete [ 3724.340903] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 3725.465072] Key type lgssc unregistered [ 3725.593310] LNet: 225726:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3725.596559] LNetError: 225726:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3725.602298] LNet: Removed LNI 192.168.204.134@tcp [ 3725.930821] Key type .llcrypt unregistered [ 3725.931940] Key type ._llcrypt unregistered [ 3735.953909] Key type ._llcrypt registered [ 3735.954957] Key type .llcrypt registered [ 3735.995329] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 3736.347599] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3736.397809] alg: No test for adler32 (adler32-zlib) [ 3737.254361] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 3737.337337] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 3738.919167] Key type lgssc registered [ 3739.297090] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3742.181404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3742.189188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3743.383833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3745.110789] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3748.667716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3748.885262] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3750.606858] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3753.792950] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3756.213425] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3758.968913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3759.115491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3761.450309] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3764.146410] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3768.359656] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 3772.229056] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3772.830902] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3772.835857] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3772.839088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.843251] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3772.843272] LustreError: 227136:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3775.456670] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3775.460342] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3775.465648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3778.533232] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3778.533697] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3778.535738] Lustre: Skipped 1 previous similar message [ 3779.611473] Lustre: server umount lustre-OST0000 complete [ 3792.919072] LustreError: 227136:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3808.089904] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3808.223587] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3810.554711] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3813.501591] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3816.127361] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 2 sec [ 3818.285430] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b200335b800.ost_server_uuid 50 [ 3818.865170] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b200335b800.ost_server_uuid in FULL state after 0 sec [ 3819.492234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.496736] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3824.612046] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3824.620910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3826.138283] Lustre: server umount lustre-OST0000 complete [ 3829.536850] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3829.538135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3829.542421] Lustre: Skipped 1 previous similar message [ 3829.547027] Lustre: Skipped 1 previous similar message [ 3833.617711] Lustre: server umount lustre-MDT0000 complete [ 3834.848240] LustreError: 227856:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3834.856775] LustreError: 227856:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3834.993724] LustreError: 227098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781687861 with bad export cookie 1558297838133729917 [ 3834.997412] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3834.999874] LustreError: 227098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3835.150988] Lustre: server umount lustre-MDT0001 complete [ 3839.218947] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 3840.331302] Key type lgssc unregistered [ 3840.459419] LNet: 232320:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3840.462647] LNetError: 232320:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3840.470344] LNet: Removed LNI 192.168.204.134@tcp [ 3840.781267] Key type .llcrypt unregistered [ 3840.782672] Key type ._llcrypt unregistered [ 3853.274517] Key type ._llcrypt registered [ 3853.275701] Key type .llcrypt registered [ 3853.316810] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 3853.764030] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3853.776303] alg: No test for adler32 (adler32-zlib) [ 3854.657300] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 3854.762332] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 3856.375432] Key type lgssc registered [ 3856.891429] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3860.770119] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3860.778698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3861.975933] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3863.591406] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3866.929638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3867.101338] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3868.719345] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3869.825751] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3872.380178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3872.528864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3874.613564] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3877.791192] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3877.873083] Lustre: lustre-OST0001: new disk, initializing [ 3877.875171] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3877.878855] Lustre: 236334:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3877.919178] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3879.966077] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3880.965047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3880.968973] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3881.008096] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3886.845448] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3887.156818] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 3892.047509] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3902.436561] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3902.438902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3902.442017] Lustre: Skipped 2 previous similar messages [ 3902.444341] Lustre: Skipped 1 previous similar message [ 3906.303825] Lustre: server umount lustre-MDT0000 complete [ 3907.554761] LustreError: 236342:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3907.561927] LustreError: 236342:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3907.858204] LustreError: 234008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781687934 with bad export cookie 5775450957025155399 [ 3907.862368] LustreError: 234764:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 3907.862443] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3907.867188] LustreError: 234008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3908.020108] Lustre: server umount lustre-MDT0001 complete [ 3919.890970] Lustre: server umount lustre-OST0000 complete [ 3930.668271] Lustre: server umount lustre-OST0001 complete [ 3938.867939] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 3942.960753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3943.126868] LustreError: 239443:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3943.152833] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3944.689509] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3947.881697] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3949.482234] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3950.549274] Lustre: 240584:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3953.124150] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3953.283875] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3953.286847] Lustre: Skipped 1 previous similar message [ 3955.493218] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3958.435663] LustreError: 240941:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3958.438261] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 3958.720239] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3961.106887] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 3963.880669] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 3967.976933] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 3974.712059] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3975.903772] Lustre: 242475:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3983.331328] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3983.332577] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3983.336687] Lustre: Skipped 4 previous similar messages [ 3983.339965] Lustre: Skipped 5 previous similar messages [ 3985.438752] Lustre: server umount mylustre-MDT0000 complete [ 3986.753348] LustreError: 242478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688013 with bad export cookie 5775450957025158220 [ 3986.757448] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3986.758072] LustreError: 242478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3986.890284] Lustre: server umount mylustre-MDT0001 complete [ 3997.641380] Lustre: server umount mylustre-OST0000 complete [ 4009.337796] Lustre: server umount mylustre-OST0001 complete [ 4017.167281] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4021.417908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4021.638847] LustreError: 244784:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4021.670698] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4021.673814] Lustre: Skipped 1 previous similar message [ 4023.319752] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4026.654655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4028.427738] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4029.587664] Lustre: 245924:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4032.320570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4034.760835] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4036.579115] LustreError: 246278:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4036.585098] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 4037.997168] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4040.413399] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4043.239784] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 4047.337444] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 4054.324517] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4055.778852] Lustre: 247813:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4062.689280] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4062.689546] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4062.692525] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4062.696912] Lustre: Skipped 1 previous similar message [ 4064.098482] Lustre: server umount tfs-MDT0000 complete [ 4065.480129] LustreError: 244760:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688091 with bad export cookie 5775450957025160397 [ 4065.483279] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4065.485111] LustreError: 244760:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4065.624820] Lustre: server umount tfs-MDT0001 complete [ 4077.062277] Lustre: server umount tfs-OST0000 complete [ 4087.704208] Lustre: server umount tfs-OST0001 complete [ 4095.470796] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4099.219917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4099.391773] LustreError: 250071:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4099.421403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4099.424716] Lustre: Skipped 3 previous similar messages [ 4100.849571] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4103.814025] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4105.384425] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4106.385063] Lustre: 251213:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4108.779047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4109.928316] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 4111.074984] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4114.023848] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 4114.250468] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4116.512344] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4119.533828] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 4119.923262] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4121.281383] Lustre: 253080:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4123.841869] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 05:22:29 (1781688149) [ 4140.005978] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4140.006944] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4140.008797] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4140.008802] Lustre: Skipped 3 previous similar messages [ 4140.015451] Lustre: Skipped 4 previous similar messages [ 4141.789272] Lustre: server umount lustre-MDT0000 complete [ 4143.008488] LustreError: 250803:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4143.015040] LustreError: 250803:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 4143.156448] LustreError: 253082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688169 with bad export cookie 5775450957025162308 [ 4143.159956] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4143.160899] LustreError: 253082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4148.193230] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4148.194423] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4148.198685] Lustre: Skipped 2 previous similar messages [ 4148.202462] Lustre: Skipped 4 previous similar messages [ 4149.361647] Lustre: server umount lustre-MDT0001 complete [ 4165.087154] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4165.124678] Lustre: server umount lustre-OST0000 complete [ 4172.575954] Lustre: server umount lustre-OST0001 complete [ 4175.197312] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 4178.792624] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4191.350658] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4191.350658] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4191.350658] [ 4191.362794] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4191.456769] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4191.472394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4191.508970] Lustre: lustre-MDT0000: new disk, initializing [ 4191.537192] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4191.540365] Lustre: Skipped 3 previous similar messages [ 4191.548752] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4193.005141] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4197.497076] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 4197.497076] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4197.497076] [ 4197.506239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4197.549116] Lustre: 256261:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4197.578146] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4197.580958] Lustre: Skipped 1 previous similar message [ 4197.617655] Lustre: lustre-MDT0001: new disk, initializing [ 4197.667475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4197.671198] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4199.191276] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4203.800895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4203.912812] Lustre: lustre-OST0000: new disk, initializing [ 4203.915178] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4203.919280] Lustre: 257846:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4205.241472] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4205.245687] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4205.280912] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4206.110074] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4210.523053] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4210.597143] Lustre: lustre-OST0001: new disk, initializing [ 4210.599684] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4210.603499] Lustre: 258719:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4211.713785] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4211.716987] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4211.741154] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4212.595122] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4216.929692] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 05:24:02 (1781688242) [ 4220.591364] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 05:24:06 (1781688246) [ 4232.166051] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4232.169547] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4232.170188] Lustre: Skipped 1 previous similar message [ 4232.175667] Lustre: Skipped 1 previous similar message [ 4238.150093] Lustre: server umount lustre-OST0000 complete [ 4242.403640] LustreError: 258043:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4242.409082] LustreError: 258043:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4245.667939] Lustre: server umount lustre-MDT0000 complete [ 4247.110610] LustreError: 256254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688273 with bad export cookie 5775450957025164086 [ 4247.114348] LustreError: 256254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4247.116057] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4252.640359] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4252.641905] Lustre: Skipped 6 previous similar messages [ 4253.301619] Lustre: server umount lustre-MDT0001 complete [ 4257.069844] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 4275.167120] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4275.237593] Lustre: server umount lustre-OST0001 complete [ 4277.258339] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 4283.362835] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4294.883334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4294.968049] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4294.981498] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4295.017256] Lustre: lustre-MDT0000: new disk, initializing [ 4295.044663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4296.314852] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4300.381916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4300.420319] Lustre: 263240:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4300.513841] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4301.870494] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4305.557127] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4307.534935] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4309.958094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4310.065504] Lustre: 265206:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4311.300243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4311.303148] Lustre: Skipped 1 previous similar message [ 4311.305386] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4311.350841] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4312.168255] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4316.032713] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4321.761188] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4321.762448] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4321.763509] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4321.763935] LustreError: Skipped 1 previous similar message [ 4321.767480] Lustre: Skipped 5 previous similar messages [ 4324.438808] Lustre: server umount lustre-OST0000 complete [ 4326.367913] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4340.191161] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4340.262909] Lustre: server umount lustre-MDT0000 complete [ 4341.635407] LustreError: 263232:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688368 with bad export cookie 5775450957025165556 [ 4341.639748] LustreError: 263232:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4341.640202] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4341.782959] Lustre: server umount lustre-MDT0001 complete [ 4345.450863] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 4346.601965] Key type lgssc unregistered [ 4346.745826] LNet: 266968:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4346.748980] LNetError: 266968:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4346.756313] LNet: Removed LNI 192.168.204.134@tcp [ 4347.062119] Key type .llcrypt unregistered [ 4347.062987] Key type ._llcrypt unregistered [ 4357.108383] Key type ._llcrypt registered [ 4357.109850] Key type .llcrypt registered [ 4357.152409] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4357.567702] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4357.598141] alg: No test for adler32 (adler32-zlib) [ 4358.454579] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 4358.538751] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 4360.119164] Key type lgssc registered [ 4360.492174] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4363.258407] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4363.265179] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4364.475735] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4366.014591] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4369.220033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4369.363666] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4370.856423] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4373.511964] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4375.317521] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4377.835856] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4377.973733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4380.028922] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4382.554675] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4382.910767] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4384.226390] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4384.227695] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4384.235936] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4385.686918] Lustre: server umount lustre-OST0000 complete [ 4389.211784] Lustre: server umount lustre-MDT0000 complete [ 4389.345231] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4389.345976] LustreError: 268401:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4389.350135] Lustre: Skipped 1 previous similar message [ 4389.357104] LustreError: 268401:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4390.574488] LustreError: 268386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688417 with bad export cookie 15257879284900040411 [ 4390.577581] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4390.578541] LustreError: 268386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4396.727073] Lustre: server umount lustre-MDT0001 complete [ 4400.445292] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 4401.607788] Key type lgssc unregistered [ 4401.740878] LNet: 271983:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4401.744465] LNetError: 271983:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4401.754538] LNet: Removed LNI 192.168.204.134@tcp [ 4402.096094] Key type .llcrypt unregistered [ 4402.097813] Key type ._llcrypt unregistered [ 4408.608197] Key type ._llcrypt registered [ 4408.609661] Key type .llcrypt registered [ 4408.656530] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4409.196554] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4409.214081] alg: No test for adler32 (adler32-zlib) [ 4410.081285] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 4410.173975] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 4411.759243] Key type lgssc registered [ 4412.159597] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4415.317791] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4415.329283] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4430.753955] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4435.874813] LustreError: 273154:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.134@tcp: failed processing log, type 4: rc = -110 [ 4462.495192] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4464.389029] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4466.768370] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4467.520320] Lustre: server umount lustre-OST0000 complete [ 4472.780045] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 4473.810730] Key type lgssc unregistered [ 4473.938364] LNet: 274781:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4473.941204] LNetError: 274781:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4473.952456] LNet: Removed LNI 192.168.204.134@tcp [ 4474.234498] Key type .llcrypt unregistered [ 4474.236042] Key type ._llcrypt unregistered [ 4487.182153] Key type ._llcrypt registered [ 4487.184157] Key type .llcrypt registered [ 4487.221219] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4487.551394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4487.576662] alg: No test for adler32 (adler32-zlib) [ 4488.431393] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 4488.511968] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 4490.095121] Key type lgssc registered [ 4490.425814] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4494.444233] loop0: detected capacity change from 0 to 204800000 [ 4494.504231] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4494.507417] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4494.620925] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4494.945146] loop0: detected capacity change from 0 to 204800000 [ 4495.000664] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4495.004413] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4495.113666] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4495.451584] loop0: detected capacity change from 0 to 204800000 [ 4495.512192] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4495.515815] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4495.552710] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4495.897991] loop0: detected capacity change from 0 to 204800000 [ 4495.959949] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4496.333165] loop0: detected capacity change from 0 to 204800000 [ 4496.346697] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4497.567316] loop0: detected capacity change from 0 to 204800000 [ 4497.578346] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4498.792474] loop0: detected capacity change from 0 to 204800000 [ 4498.800878] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4500.020621] loop0: detected capacity change from 0 to 204800000 [ 4500.034531] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4501.253739] loop0: detected capacity change from 0 to 204800000 [ 4501.280754] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4501.288851] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4501.322087] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4501.325499] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4501.328051] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4501.330809] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4501.333491] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4501.336055] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4501.338868] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4501.342172] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4501.347532] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4501.351377] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4501.354615] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4501.358895] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4501.362277] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4501.365804] Lustre: 278173:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4503.838795] Lustre: server umount MGS complete [ 4504.109560] loop0: detected capacity change from 0 to 204800000 [ 4504.133284] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4504.302830] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4504.320517] Lustre: lustre-MDT0000: Aborting client recovery [ 4504.605385] loop1: detected capacity change from 0 to 204800000 [ 4504.633793] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4504.681846] Lustre: 278603:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4504.687513] Lustre: 278603:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4504.691447] Lustre: 278603:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4504.695024] Lustre: 278603:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4504.698166] Lustre: 278603:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4504.702018] Lustre: 278603:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4504.705736] Lustre: 278603:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4504.712434] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4505.102711] loop2: detected capacity change from 0 to 204800000 [ 4505.129440] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4505.159407] Lustre: 278684:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4505.166810] Lustre: 278684:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4505.172079] Lustre: 278684:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4505.176248] Lustre: 278684:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4505.179354] Lustre: 278684:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4505.182970] Lustre: 278684:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4505.186601] Lustre: 278684:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4505.241079] Lustre: Found index 0 for lustre-OST0000, updating log [ 4505.253168] Lustre: 278702:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4505.273104] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4505.274907] Lustre: Skipped 1 previous similar message [ 4505.280785] Lustre: lustre-OST0000: Aborting recovery [ 4505.560838] loop3: detected capacity change from 0 to 204800000 [ 4505.592696] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4505.624201] Lustre: 278767:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4505.627588] Lustre: 278767:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4505.630680] Lustre: 278767:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4505.633940] Lustre: 278767:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4505.637137] Lustre: 278767:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4505.640490] Lustre: 278767:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4505.643483] Lustre: 278767:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4513.825747] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4513.826199] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4513.835058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4513.838819] LustreError: 278547:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 4513.839903] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4514.850055] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4514.850634] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4515.897762] LustreError: 278514:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 225us: rc = -2 [ 4530.145084] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4530.145908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4530.150090] Lustre: Skipped 2 previous similar messages [ 4530.151791] Lustre: Skipped 1 previous similar message [ 4533.143306] Lustre: server umount lustre-MDT0000 complete [ 4533.432715] LustreError: 278610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688559 with bad export cookie 9685760853659620398 [ 4533.434489] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4533.436855] LustreError: 278610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4535.263534] LustreError: 278511:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4535.263840] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4535.269435] LustreError: 278511:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4535.272781] Lustre: Skipped 1 previous similar message [ 4535.273291] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4535.280346] Lustre: Skipped 3 previous similar messages [ 4538.271496] LustreError: 278512:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4538.271601] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4538.277246] LustreError: 278512:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4538.279861] Lustre: Skipped 1 previous similar message [ 4539.634923] Lustre: server umount lustre-MDT0001 complete [ 4554.207116] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4554.250702] Lustre: server umount lustre-OST0000 complete [ 4560.662187] Lustre: server umount lustre-OST0001 complete [ 4563.059434] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 05:29:49 (1781688589) [ 4572.015241] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 4574.993681] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4586.565917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4586.661119] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4586.672380] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4586.711584] Lustre: lustre-MDT0000: new disk, initializing [ 4586.736543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4586.738614] Lustre: Skipped 1 previous similar message [ 4586.745313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4588.036850] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4592.025439] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4592.064848] Lustre: 282831:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4592.085821] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4592.087605] Lustre: Skipped 1 previous similar message [ 4592.126103] Lustre: lustre-MDT0001: new disk, initializing [ 4592.157054] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4592.168682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4592.171971] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4593.521733] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4597.124518] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4598.861913] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4601.123244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4601.233167] Lustre: lustre-OST0000: new disk, initializing [ 4601.235141] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4601.238249] Lustre: 284798:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4601.240739] Lustre: 284798:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 4601.269630] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4602.298980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4602.302889] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4602.334720] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4603.254419] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4606.797894] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4609.075873] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4611.481511] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4617.284451] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4658.657118] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4658.658397] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4658.659949] LustreError: Skipped 1 previous similar message [ 4658.663968] Lustre: Skipped 1 previous similar message [ 4658.665699] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4658.669045] Lustre: Skipped 1 previous similar message [ 4659.795870] Lustre: server umount lustre-OST0000 complete [ 4663.777666] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4663.777751] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4663.778760] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4663.783117] Lustre: Skipped 2 previous similar messages [ 4667.181792] Lustre: server umount lustre-MDT0000 complete [ 4668.419016] LustreError: 282823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688694 with bad export cookie 9685760853659623107 [ 4668.422700] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4668.422793] LustreError: 282823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4668.544701] Lustre: server umount lustre-MDT0001 complete [ 4671.418456] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4672.849123] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4674.573227] Lustre: server umount MGS complete [ 4678.309992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4678.477120] LustreError: 288608:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4678.483524] LustreError: 288621:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4678.505027] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4679.891277] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4682.701449] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4684.272276] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4686.711225] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4688.491392] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4690.766607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4692.851865] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4695.264234] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4724.706951] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4724.707060] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4724.707740] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.707744] Lustre: Skipped 1 previous similar message [ 4724.719419] Lustre: Skipped 1 previous similar message [ 4729.876089] Lustre: server umount lustre-OST0000 complete [ 4733.919868] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4733.922045] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4737.308887] Lustre: server umount lustre-MDT0000 complete [ 4738.595440] LustreError: 291352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688765 with bad export cookie 9685760853659624913 [ 4738.597769] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4738.599993] LustreError: 291352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4738.722696] Lustre: server umount lustre-MDT0001 complete [ 4742.411798] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 4743.536208] Key type lgssc unregistered [ 4743.682421] LNet: 292569:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4743.685822] LNetError: 292569:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4743.693273] LNet: Removed LNI 192.168.204.134@tcp [ 4744.006703] Key type .llcrypt unregistered [ 4744.008051] Key type ._llcrypt unregistered [ 4756.162401] Key type ._llcrypt registered [ 4756.163541] Key type .llcrypt registered [ 4756.210692] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 4762.519169] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4762.893916] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4762.901761] alg: No test for adler32 (adler32-zlib) [ 4763.775888] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 4763.865487] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 4765.455234] Key type lgssc registered [ 4765.850889] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4776.889526] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4776.895701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4777.994420] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4778.013764] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4778.051248] Lustre: lustre-MDT0000: new disk, initializing [ 4778.075104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4778.081581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4779.405917] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4783.502253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4783.547072] Lustre: 296498:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4783.583896] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4783.584815] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4783.587587] Lustre: Skipped 1 previous similar message [ 4783.628970] Lustre: lustre-MDT0001: new disk, initializing [ 4783.662434] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4783.675783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4783.679245] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4785.080279] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4788.876658] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4790.737757] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4793.089395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4793.215844] Lustre: lustre-OST0000: new disk, initializing [ 4793.217982] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4793.220766] Lustre: 298463:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4793.250499] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4795.193628] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4798.916829] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4799.480391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4799.485183] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4799.520087] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4807.436924] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4816.407303] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4824.940918] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4829.457488] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4881.380847] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4881.381317] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4881.382570] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4881.385402] Lustre: Skipped 1 previous similar message [ 4881.387729] LustreError: Skipped 1 previous similar message [ 4884.500213] Lustre: server umount lustre-OST0000 complete [ 4885.983809] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4885.986267] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4885.990905] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4885.994147] Lustre: Skipped 1 previous similar message [ 4891.616247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4891.617886] Lustre: Skipped 3 previous similar messages [ 4896.736679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4896.738415] Lustre: Skipped 1 previous similar message [ 4900.319140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4900.386692] Lustre: server umount lustre-MDT0000 complete [ 4901.629279] LustreError: 300476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781688928 with bad export cookie 13733774767766053418 [ 4901.631895] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4901.635200] LustreError: 300476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4901.764911] Lustre: server umount lustre-MDT0001 complete [ 4904.741237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4906.194589] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4907.863707] Lustre: server umount MGS complete [ 4911.636346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4911.791360] LustreError: 302618:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4911.798896] LustreError: 302631:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4911.799483] LustreError: 302618:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4911.824977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4913.219881] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4916.155336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4916.286244] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4917.722780] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4920.149555] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4922.066972] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4924.416590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4924.548199] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4926.528029] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 4928.698487] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4961.759920] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4961.763961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4961.770572] Lustre: Skipped 1 previous similar message [ 4961.773552] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4966.160531] Lustre: server umount lustre-OST0000 complete [ 4967.904295] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4967.909973] Lustre: Skipped 1 previous similar message [ 4972.511992] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4972.515480] LustreError: Skipped 1 previous similar message [ 4972.517549] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4973.590850] Lustre: server umount lustre-MDT0000 complete [ 4974.859658] LustreError: 305364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689001 with bad export cookie 13733774767766055273 [ 4974.863568] LustreError: 305364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4974.864455] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4974.999640] Lustre: server umount lustre-MDT0001 complete [ 4978.613034] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 4979.577355] Key type lgssc unregistered [ 4979.704331] LNet: 306582:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4979.707617] LNetError: 306582:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4979.718280] LNet: Removed LNI 192.168.204.134@tcp [ 4980.010432] Key type .llcrypt unregistered [ 4980.011482] Key type ._llcrypt unregistered [ 4986.480034] Key type ._llcrypt registered [ 4986.480986] Key type .llcrypt registered [ 4986.515431] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 4992.093552] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 4992.413282] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4992.483591] alg: No test for adler32 (adler32-zlib) [ 4993.338250] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 4993.411945] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 4994.983155] Key type lgssc registered [ 4995.303443] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5002.559992] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5007.204168] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5007.210936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5008.328729] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5008.341522] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5008.380677] Lustre: lustre-MDT0000: new disk, initializing [ 5008.405763] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5008.414144] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5009.820290] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5014.101540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5014.228353] Lustre: lustre-OST0000: new disk, initializing [ 5014.230852] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5014.232848] Lustre: Skipped 1 previous similar message [ 5014.235842] Lustre: 310605:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5014.262830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5015.558937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5015.562839] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5015.571614] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5016.400786] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5189.600366] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5189.606230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5194.720669] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5194.845521] Lustre: server umount lustre-MDT0000 complete [ 5196.092581] LustreError: 310604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689222 with bad export cookie 7662142079755500766 [ 5196.095828] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5196.097151] LustreError: 310604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5196.163570] Lustre: server umount lustre-OST0000 complete [ 5204.548340] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 5207.978537] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5220.155489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5220.252649] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5220.267590] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5220.309180] Lustre: lustre-MDT0000: new disk, initializing [ 5220.339334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5220.346761] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5221.707361] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5225.997539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5226.049283] Lustre: 314828:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5226.080088] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5226.082996] Lustre: Skipped 1 previous similar message [ 5226.128123] Lustre: lustre-MDT0001: new disk, initializing [ 5226.162403] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5226.175155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5226.179197] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5227.567863] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5231.363750] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5233.164801] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5235.426340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5235.547942] Lustre: lustre-OST0000: new disk, initializing [ 5235.549686] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5235.552627] Lustre: 316796:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5235.587145] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5236.662691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5236.666330] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5236.699486] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5237.586734] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5241.253476] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5242.710360] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5242.792545] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5244.173775] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5244.264768] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5246.949771] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5246.951488] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5246.953356] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5246.954657] Lustre: Skipped 1 previous similar message [ 5246.957374] LustreError: Skipped 1 previous similar message [ 5252.066073] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5252.069064] Lustre: Skipped 1 previous similar message [ 5257.190402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5257.193521] Lustre: Skipped 1 previous similar message [ 5259.231123] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5259.295509] Lustre: server umount lustre-OST0000 complete [ 5262.305113] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5262.305312] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5262.308351] Lustre: Skipped 1 previous similar message [ 5266.715598] Lustre: server umount lustre-MDT0000 complete [ 5268.012403] LustreError: 314821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689294 with bad export cookie 7662142079758441543 [ 5268.015535] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5268.015611] LustreError: 314821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5268.148619] Lustre: server umount lustre-MDT0001 complete [ 5269.240084] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5269.759893] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 05:41:35 (1781689295) [ 5272.102564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5272.285517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5273.757257] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5276.668415] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5278.470562] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5280.998930] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5282.815807] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5285.250308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5287.424675] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5289.745186] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5291.866395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5291.940536] Lustre: lustre-OST0001: new disk, initializing [ 5291.942210] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5291.944552] Lustre: 321940:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5291.981118] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5291.983053] Lustre: Skipped 2 previous similar messages [ 5293.801918] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5295.805504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5295.808628] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5295.822058] LustreError: 322470:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5296.869067] LustreError: 322478:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5296.872966] LustreError: 322478:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5297.436153] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5297.504899] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5297.892895] LustreError: 322470:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5297.896677] LustreError: 322470:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5299.240474] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b202cf97800.ost_server_uuid 50 [ 5299.756254] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b202cf97800.ost_server_uuid in FULL state after 0 sec [ 5299.938421] LustreError: 322478:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5299.943130] LustreError: 322478:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5301.572921] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8b202cf97800.ost_server_uuid 50 [ 5302.056179] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8b202cf97800.ost_server_uuid in FULL state after 0 sec [ 5303.018482] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5316.695272] Lustre: lustre-OST0001: Not available for connect from 192.168.204.34@tcp (stopping) [ 5316.697440] Lustre: Skipped 2 previous similar messages [ 5317.604866] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5317.608750] Lustre: Skipped 1 previous similar message [ 5328.351110] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5328.394298] Lustre: server umount lustre-OST0001 complete [ 5332.961728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5332.963149] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5332.963377] LustreError: 320928:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5332.963388] LustreError: 320928:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 5332.965638] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5332.968406] Lustre: Skipped 1 previous similar message [ 5332.977922] Lustre: Skipped 9 previous similar messages [ 5336.208789] Lustre: server umount lustre-OST0000 complete [ 5343.599082] Lustre: server umount lustre-MDT0000 complete [ 5344.815293] LustreError: 319090:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689371 with bad export cookie 7662142079758442411 [ 5344.819385] LustreError: 319090:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5344.819798] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5344.943289] Lustre: server umount lustre-MDT0001 complete [ 5348.466426] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 5349.475580] Key type lgssc unregistered [ 5349.612399] LNet: 324627:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5349.615280] LNetError: 324627:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5349.629306] LNet: Removed LNI 192.168.204.134@tcp [ 5349.931110] Key type .llcrypt unregistered [ 5349.932226] Key type ._llcrypt unregistered [ 5359.734758] Key type ._llcrypt registered [ 5359.735697] Key type .llcrypt registered [ 5359.769840] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5360.088114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5360.139656] alg: No test for adler32 (adler32-zlib) [ 5361.002625] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 5361.096535] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 5362.687193] Key type lgssc registered [ 5363.071907] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5365.931420] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5365.937314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5367.124798] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5368.839530] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5371.528288] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5374.046963] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5374.216546] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5375.748114] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5379.034310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5379.168922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5381.268855] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5383.661348] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5384.296979] LustreError: 327702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5384.302552] LustreError: 327702:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5386.015793] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5386.112603] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5388.121206] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5390.982672] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5392.364721] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5392.366700] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5398.004871] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5398.095494] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5478.368177] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5478.371150] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5478.377396] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5479.394339] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5479.396667] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5479.397906] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5479.401644] Lustre: Skipped 1 previous similar message [ 5484.518731] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5484.520736] Lustre: Skipped 1 previous similar message [ 5489.119127] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5489.154924] Lustre: server umount lustre-OST0001 complete [ 5489.638988] LustreError: 329734:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5489.643498] LustreError: 329734:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5492.193119] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5492.195761] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5492.199853] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5494.757204] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5494.758433] LustreError: 328429:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5494.758619] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5494.766589] LustreError: 328429:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5499.876311] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5499.876390] LustreError: 328429:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5499.878171] Lustre: Skipped 3 previous similar messages [ 5499.886565] LustreError: 328429:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5504.991103] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5504.993424] LustreError: 328427:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5504.999647] LustreError: 328427:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5505.058436] Lustre: server umount lustre-OST0000 complete [ 5507.552482] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5507.558340] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5510.113290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5510.120130] Lustre: Skipped 4 previous similar messages [ 5512.768882] Lustre: server umount lustre-MDT0000 complete [ 5514.210404] LustreError: 326005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689540 with bad export cookie 3450687341780048505 [ 5514.214261] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5514.214748] LustreError: 326005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5514.348041] Lustre: server umount lustre-MDT0001 complete [ 5518.157106] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 5519.298278] Key type lgssc unregistered [ 5519.424389] LNet: 330962:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5519.427412] LNetError: 330962:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5519.438310] LNet: Removed LNI 192.168.204.134@tcp [ 5519.739448] Key type .llcrypt unregistered [ 5519.740520] Key type ._llcrypt unregistered [ 5537.889552] Key type ._llcrypt registered [ 5537.890610] Key type .llcrypt registered [ 5537.927135] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5538.279234] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5538.324060] alg: No test for adler32 (adler32-zlib) [ 5539.189381] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 5539.277761] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 5540.863188] Key type lgssc registered [ 5541.319313] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5544.584654] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5544.590833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5545.752506] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5547.101876] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5549.919106] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5550.046452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5551.336815] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5552.355198] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5554.852828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5555.042668] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5556.066920] LustreError: 334715:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5556.070650] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5556.072699] LustreError: 334715:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5556.966577] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5560.159085] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5560.167305] LustreError: 334903:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5560.172583] LustreError: 334903:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5560.252409] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5562.321712] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5565.428843] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5566.546645] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5571.349569] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5575.654011] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5575.657254] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5575.657740] Lustre: Skipped 3 previous similar messages [ 5575.661176] Lustre: Skipped 2 previous similar messages [ 5575.664615] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5580.771985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5580.774150] Lustre: Skipped 1 previous similar message [ 5581.029171] Lustre: server umount lustre-MDT0000 complete [ 5582.428631] LustreError: 333208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689608 with bad export cookie 14972018293084474078 [ 5582.432028] LustreError: 333208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5582.432137] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5582.568374] Lustre: server umount lustre-MDT0001 complete [ 5594.131522] Lustre: server umount lustre-OST0000 complete [ 5606.386406] Lustre: server umount lustre-OST0001 complete [ 5616.168122] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5623.536680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5623.771030] LustreError: 338488:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5623.813835] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5625.757107] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5628.896250] LustreError: 338489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5630.264588] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5632.428093] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5633.835262] Lustre: 339627:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5636.976286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5637.182842] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5637.189849] Lustre: Skipped 1 previous similar message [ 5639.949361] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5640.230702] LustreError: 339982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5643.500557] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5645.843416] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5648.749173] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5648.756324] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5654.774677] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5656.343080] Lustre: 341496:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5659.111481] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5659.113244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5659.117775] Lustre: Skipped 3 previous similar messages [ 5659.120704] Lustre: Skipped 2 previous similar messages [ 5663.714567] Lustre: server umount lustre-MDT0000 complete [ 5664.224175] LustreError: 338488:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5664.231112] LustreError: 338488:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5665.164193] LustreError: 338470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689691 with bad export cookie 14972018293084475737 [ 5665.166612] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5665.169115] LustreError: 338470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5665.306319] Lustre: server umount lustre-MDT0001 complete [ 5677.579352] Lustre: server umount lustre-OST0000 complete [ 5689.242507] Lustre: server umount lustre-OST0001 complete [ 5690.503707] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5691.096891] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5691.709963] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 05:48:37 (1781689717) [ 5699.496107] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5702.915054] loop0: detected capacity change from 0 to 1099511627776 [ 5703.065079] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5703.067888] *** VALIDATE xfs *** [ 5703.071989] XFS (loop0): Mounting V5 Filesystem [ 5703.081523] XFS (loop0): Ending clean mount [ 5703.514560] loop1: detected capacity change from 0 to 18691697672192 [ 5724.670620] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 5727.647099] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5739.806955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5739.923251] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5739.937797] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5739.987279] Lustre: lustre-MDT0000: new disk, initializing [ 5740.012968] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5740.016087] Lustre: Skipped 1 previous similar message [ 5740.022053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5741.382639] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5745.582387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5745.621229] Lustre: 346607:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5745.643625] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5745.645473] Lustre: Skipped 1 previous similar message [ 5745.681449] Lustre: lustre-MDT0001: new disk, initializing [ 5745.722940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5745.727653] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5747.096436] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5751.004929] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5752.853718] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5755.107324] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5755.218064] Lustre: lustre-OST0000: new disk, initializing [ 5755.219518] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5755.221610] Lustre: 348576:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5756.737304] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5756.740557] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5756.786688] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5757.173620] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5760.977561] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5762.448120] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5762.534118] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5763.876780] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5763.949623] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5766.674474] Lustre: server umount lustre-OST0000 complete [ 5771.231960] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5771.234330] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5771.239452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5771.241987] Lustre: Skipped 1 previous similar message [ 5777.376093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5777.377693] Lustre: Skipped 3 previous similar messages [ 5778.430444] Lustre: server umount lustre-MDT0000 complete [ 5779.687221] LustreError: 346599:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689806 with bad export cookie 14972018293084477270 [ 5779.690478] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5779.690786] LustreError: 346599:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5790.223636] Lustre: server umount lustre-MDT0001 complete [ 5792.853536] XFS (loop0): Unmounting Filesystem [ 5794.488488] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 05:50:20 (1781689820) [ 5796.734680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5796.903330] LustreError: 351132:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5796.928664] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5796.930704] Lustre: Skipped 2 previous similar messages [ 5798.287290] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5801.179070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5802.835904] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5805.389645] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5807.310145] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5809.687271] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5812.009110] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5814.421482] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5828.577828] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5828.578063] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5828.578748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5828.587663] Lustre: Skipped 2 previous similar messages [ 5834.324582] Lustre: server umount lustre-OST0000 complete [ 5837.279931] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5837.282855] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5841.689049] Lustre: server umount lustre-MDT0000 complete [ 5842.932495] LustreError: 351114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781689869 with bad export cookie 14972018293084478152 [ 5842.937183] LustreError: 351114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5842.937950] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5843.062938] Lustre: server umount lustre-MDT0001 complete [ 5846.648901] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 5847.753294] Key type lgssc unregistered [ 5847.893574] LNet: 354780:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5847.896578] LNetError: 354780:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5847.904323] LNet: Removed LNI 192.168.204.134@tcp [ 5848.222153] Key type .llcrypt unregistered [ 5848.223176] Key type ._llcrypt unregistered [ 5858.191280] Key type ._llcrypt registered [ 5858.192347] Key type .llcrypt registered [ 5858.231724] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 5858.711286] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5858.718654] alg: No test for adler32 (adler32-zlib) [ 5859.581921] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 5859.668408] LNet: Added LNI 192.168.204.134@tcp [8/256/0/180] [ 5861.247161] Key type lgssc registered [ 5861.607908] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5864.093752] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5864.100726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5865.284791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5866.543414] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5869.235506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5869.356600] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5870.598831] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5872.734463] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5874.282469] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5876.544254] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5876.680437] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5878.711071] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5881.182093] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5895.135809] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5895.138972] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5895.145419] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5896.160211] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5896.162715] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5900.370927] Lustre: server umount lustre-MDT0001 complete [ 5901.281191] LustreError: 356322:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5901.287583] LustreError: 356322:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5901.900470] LustreError: 356317:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5903.664208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5903.699546] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5903.701037] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5903.712493] Lustre: 356310:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5903.820485] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5905.122851] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 5908.964923] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5913.061974] LustreError: 360159:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5913.065185] LustreError: 360159:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5913.068041] LustreError: 356314:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5913.087449] LustreError: 360167:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.134@tcp: cfg command failed: rc = -17 [ 5913.090333] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5913.090333] [ 6209.108986] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.34@tcp (stopping) [ 6209.113091] Lustre: Skipped 2 previous similar messages [ 6211.041551] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6211.043132] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6211.049172] Lustre: Skipped 1 previous similar message [ 6214.220931] LustreError: 356316:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6214.241054] Lustre: server umount lustre-MDT0001 complete [ 6217.600087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6217.638473] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6217.640034] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6217.656181] Lustre: 356309:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6217.789345] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6219.218866] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 6222.823187] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6225.382695] LustreError: 372731:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6225.386147] LustreError: 372731:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6225.390363] LustreError: 356314:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6225.392281] LustreError: 356314:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6225.411200] LustreError: 372734:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6225.414868] LustreError: 372734:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6225.416647] LustreError: 372734:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.134@tcp: cfg command failed: rc = -17 [ 6225.420973] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.134@tcp [ 6225.420973] [ 6524.897792] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6524.898194] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6524.899522] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6524.899528] Lustre: Skipped 2 previous similar messages [ 6524.910373] Lustre: Skipped 1 previous similar message [ 6530.017898] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6530.021453] Lustre: Skipped 2 previous similar messages [ 6536.159179] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6536.228224] Lustre: server umount lustre-MDT0001 complete [ 6536.780389] LustreError: 356316:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6536.789031] LustreError: 356316:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6539.763463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6539.800694] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6539.802746] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6539.819599] Lustre: 356309:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6539.936235] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6541.311244] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 6545.388383] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6549.477780] LustreError: 385468:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6549.482315] LustreError: 385468:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6549.487391] LustreError: 356314:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6549.489523] LustreError: 356314:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6549.505050] LustreError: 385471:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6549.509779] LustreError: 385471:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.134@tcp: cfg command failed: rc = -114 [ 6549.512220] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6549.512220] [ 6857.698037] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6857.699092] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6857.700043] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6857.700046] Lustre: Skipped 3 previous similar messages [ 6857.702121] Lustre: Skipped 1 previous similar message [ 6861.409433] Lustre: server umount lustre-MDT0000 complete [ 6862.718077] LustreError: 356303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781690889 with bad export cookie 9502980057206586435 [ 6862.721407] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6862.722302] LustreError: 356303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6862.815952] LustreError: 358509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6873.056048] LustreError: 358509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6873.060141] LustreError: 358509:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6877.151108] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6877.255266] Lustre: server umount lustre-MDT0001 complete [ 6878.651918] Lustre: server umount lustre-OST0000 complete [ 6881.732298] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 6885.159004] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 6897.189828] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 06:08:43 (1781690923) [ 6899.827105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6899.932260] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6899.947961] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6899.991704] Lustre: lustre-MDT0000: new disk, initializing [ 6900.016678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6900.025379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6901.364471] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 6906.350850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6906.396918] Lustre: 399931:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6906.423269] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6906.425309] Lustre: Skipped 1 previous similar message [ 6906.467897] Lustre: lustre-MDT0001: new disk, initializing [ 6906.500352] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6906.513037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6906.517227] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6907.754421] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 6911.420749] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6913.206786] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6915.352186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6915.454671] Lustre: lustre-OST0000: new disk, initializing [ 6915.456348] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6915.458961] Lustre: 401899:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6915.488138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6917.387300] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_default_debug -1 all [ 6918.968427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6918.974098] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6919.004763] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6921.193667] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6924.257789] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6924.258592] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6924.261072] Lustre: Skipped 3 previous similar messages [ 6924.263794] Lustre: Skipped 7 previous similar messages [ 6929.972162] Lustre: server umount lustre-MDT0000 complete [ 6931.284494] LustreError: 399923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781690957 with bad export cookie 9502980057206588374 [ 6931.287529] LustreError: 400737:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6931.287986] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6931.289814] LustreError: 399923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6931.401483] Lustre: server umount lustre-MDT0001 complete [ 6943.087913] Lustre: server umount lustre-OST0000 complete [ 6952.778562] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing set_hostid [ 6955.879298] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing load_modules_local [ 6966.394936] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6855 sec ========== 06:09:52 (1781690992) [ 6966.924736] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 06:09:52 (1781690992) === [ 6968.025481] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 06:09:54 (1781690994) === [ 6982.087462] Lustre: DEBUG MARKER: oleg434-server.virtnet: executing unload_modules_local [ 6983.259666] Key type lgssc unregistered [ 6983.393678] LNet: 408252:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6983.398550] LNetError: 408252:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6983.411399] LNet: Removed LNI 192.168.204.134@tcp [ 6983.710785] Key type .llcrypt unregistered [ 6983.712632] Key type ._llcrypt unregistered