[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 532383045 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003327] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008688] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009018] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010011] pid_max: default: 32768 minimum: 301 [ 0.011129] LSM: Security Framework initializing [ 0.013013] Yama: becoming mindful. [ 0.014034] SELinux: Initializing. [ 0.015067] *** VALIDATE selinux *** [ 0.023201] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027775] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028149] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029125] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030141] *** VALIDATE tmpfs *** [ 0.031485] *** VALIDATE proc *** [ 0.032250] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.034271] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035148] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037028] Spectre V2 : User space: Vulnerable [ 0.038009] Speculative Store Bypass: Vulnerable [ 0.040978] debug: unmapping init [mem 0xffffffffb2859000-0xffffffffb2860fff] [ 0.043872] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044619] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045024] ... version: 2 [ 0.046013] ... bit width: 48 [ 0.047013] ... generic registers: 4 [ 0.048014] ... value mask: 0000ffffffffffff [ 0.049014] ... max period: 00007fffffffffff [ 0.050017] ... fixed-purpose events: 3 [ 0.051012] ... event mask: 000000070000000f [ 0.052290] rcu: Hierarchical SRCU implementation. [ 0.054412] smp: Bringing up secondary CPUs ... [ 0.055595] x86: Booting SMP configuration: [ 0.056018] .... node #0, CPUs: #1 #2 #3 [ 0.059254] smp: Brought up 1 node, 4 CPUs [ 0.061013] smpboot: Max logical packages: 1 [ 0.062016] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.222033] node 0 deferred pages initialised in 158ms [ 0.226129] devtmpfs: initialized [ 0.227202] x86/mm: Memory block size: 128MB [ 0.229000] gcov: version magic: 0x41383552 [ 0.232166] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.236085] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.238288] pinctrl core: initialized pinctrl subsystem [ 0.241164] [ 0.241812] ************************************************************* [ 0.244016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.247016] ** ** [ 0.249012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.252014] ** ** [ 0.254012] ** This means that this kernel is built to expose internal ** [ 0.257015] ** IOMMU data structures, which may compromise security on ** [ 0.260012] ** your system. ** [ 0.262023] ** ** [ 0.265013] ** If you see this message and you are not debugging the ** [ 0.267012] ** kernel, report this immediately to your vendor! ** [ 0.270013] ** ** [ 0.273013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.275013] ************************************************************* [ 0.278634] NET: Registered protocol family 16 [ 0.280438] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.284061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.286060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.290043] cpuidle: using governor menu [ 0.291836] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.295090] PCI: Using configuration type 1 for base access [ 0.297162] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.305082] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.307098] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.311096] cryptd: max_cpu_qlen set to 1000 [ 0.315055] ACPI: Added _OSI(Module Device) [ 0.316013] ACPI: Added _OSI(Processor Device) [ 0.317011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.318031] ACPI: Added _OSI(Processor Aggregator Device) [ 0.323825] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.330632] ACPI: Interpreter enabled [ 0.332074] ACPI: PM: (supports S0 S3 S4 S5) [ 0.334016] ACPI: Using IOAPIC for interrupt routing [ 0.335158] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.339432] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.350295] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.352050] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.354027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.357107] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.363682] acpiphp: Slot [2] registered [ 0.364189] acpiphp: Slot [5] registered [ 0.366218] acpiphp: Slot [6] registered [ 0.368212] acpiphp: Slot [7] registered [ 0.370190] acpiphp: Slot [8] registered [ 0.371178] acpiphp: Slot [9] registered [ 0.373243] acpiphp: Slot [10] registered [ 0.375182] acpiphp: Slot [3] registered [ 0.377133] acpiphp: Slot [4] registered [ 0.378127] acpiphp: Slot [11] registered [ 0.380119] acpiphp: Slot [12] registered [ 0.382148] acpiphp: Slot [13] registered [ 0.384179] acpiphp: Slot [14] registered [ 0.385141] acpiphp: Slot [15] registered [ 0.387144] acpiphp: Slot [16] registered [ 0.389124] acpiphp: Slot [17] registered [ 0.391135] acpiphp: Slot [18] registered [ 0.392164] acpiphp: Slot [19] registered [ 0.394205] acpiphp: Slot [20] registered [ 0.396145] acpiphp: Slot [21] registered [ 0.397163] acpiphp: Slot [22] registered [ 0.399142] acpiphp: Slot [23] registered [ 0.400112] acpiphp: Slot [24] registered [ 0.402177] acpiphp: Slot [25] registered [ 0.403126] acpiphp: Slot [26] registered [ 0.405158] acpiphp: Slot [27] registered [ 0.406117] acpiphp: Slot [28] registered [ 0.408125] acpiphp: Slot [29] registered [ 0.409187] acpiphp: Slot [30] registered [ 0.410126] acpiphp: Slot [31] registered [ 0.412085] PCI host bridge to bus 0000:00 [ 0.413024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.416036] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.418029] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.421023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.423029] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.427028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.429245] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.432011] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.435518] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.446020] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.453064] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.456026] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.458022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.461030] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.464817] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.468177] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.472060] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.475981] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.481015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.494016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.501014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.508217] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.518024] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.526021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.543018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.561434] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.572016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.588034] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.625016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.640521] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.650018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.656021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.679019] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.692012] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.705017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.714015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.742022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.756695] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.763019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.769018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.786022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.795770] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.811013] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.816015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.828015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.840803] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.843406] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.845394] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.848474] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.851263] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.855155] iommu: Default domain type: Passthrough [ 0.857553] SCSI subsystem initialized [ 0.859158] ACPI: bus type USB registered [ 0.860175] usbcore: registered new interface driver usbfs [ 0.862115] usbcore: registered new interface driver hub [ 0.864126] usbcore: registered new device driver usb [ 0.866232] pps_core: LinuxPPS API ver. 1 registered [ 0.868029] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.871061] PTP clock support registered [ 0.874045] EDAC MC: Ver: 3.0.0 [ 0.876203] PCI: Using ACPI for IRQ routing [ 0.877830] NetLabel: Initializing [ 0.879014] NetLabel: domain hash size = 128 [ 0.881013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.883084] NetLabel: unlabeled traffic allowed by default [ 0.885138] vgaarb: loaded [ 0.887171] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.889011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.894330] clocksource: Switched to clocksource kvm-clock [ 1.015277] VFS: Disk quotas dquot_6.6.0 [ 1.018199] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.020527] *** VALIDATE ramfs *** [ 1.021702] *** VALIDATE hugetlbfs *** [ 1.023380] pnp: PnP ACPI init [ 1.025695] pnp: PnP ACPI: found 6 devices [ 1.046539] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.049599] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.051484] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.053416] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.055565] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.057717] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.060277] NET: Registered protocol family 2 [ 1.062605] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.066951] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.070301] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.075177] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.078435] TCP: Hash tables configured (established 65536 bind 65536) [ 1.081187] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.084151] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.086835] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.089641] NET: Registered protocol family 1 [ 1.092234] RPC: Registered named UNIX socket transport module. [ 1.094497] RPC: Registered udp transport module. [ 1.096729] RPC: Registered tcp transport module. [ 1.098660] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.101174] NET: Registered protocol family 44 [ 1.103049] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.105294] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.107122] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.109147] PCI: CLS 0 bytes, default 64 [ 1.111171] Unpacking initramfs... [ 2.525055] debug: unmapping init [mem 0xffff922dbcc54000-0xffff922dbffbffff] [ 2.529461] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.532196] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.535183] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.056921] Initialise system trusted keyrings [ 3.058863] Key type blacklist registered [ 3.062587] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.077854] zbud: loaded [ 3.081183] *** VALIDATE nfs *** [ 3.082616] *** VALIDATE nfs4 *** [ 3.084419] pstore: using deflate compression [ 3.090044] Platform Keyring initialized [ 3.230680] NET: Registered protocol family 38 [ 3.232150] Key type asymmetric registered [ 3.233157] Asymmetric key parser 'x509' registered [ 3.234446] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.236457] io scheduler mq-deadline registered [ 3.237454] io scheduler kyber registered [ 3.238396] io scheduler bfq registered [ 3.240234] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.242135] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.244252] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.246095] ACPI: Power Button [PWRF] [ 3.252783] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.259667] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.280507] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.288992] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.306725] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.333117] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.359840] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.365686] Non-volatile memory driver v1.3 [ 3.366977] Linux agpgart interface v0.103 [ 3.395981] virtio_blk virtio1: [vda] 146648 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.398553] vda: detected capacity change from 0 to 75083776 [ 3.412925] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.415215] vdb: detected capacity change from 0 to 1073741824 [ 3.429119] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.431494] vdc: detected capacity change from 0 to 2621440000 [ 3.450371] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.452686] vdd: detected capacity change from 0 to 2621440000 [ 3.467609] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.470087] vde: detected capacity change from 0 to 4294967296 [ 3.485627] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.488044] vdf: detected capacity change from 0 to 4294967296 [ 3.499237] libphy: Fixed MDIO Bus: probed [ 3.507603] usbcore: registered new interface driver usbserial_generic [ 3.509906] usbserial: USB Serial support registered for generic [ 3.511644] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.515627] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.517078] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.520487] mousedev: PS/2 mouse device common for all mice [ 3.522924] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.523635] rtc_cmos 00:05: RTC can wake from S4 [ 3.528830] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.530744] rtc_cmos 00:05: registered as rtc0 [ 3.532526] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.534345] intel_pstate: CPU model not supported [ 3.535863] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.538805] hid: raw HID events driver (C) Jiri Kosina [ 3.540223] usbcore: registered new interface driver usbhid [ 3.541506] usbhid: USB HID core driver [ 3.542443] drop_monitor: Initializing network drop monitor service [ 3.544348] Initializing XFRM netlink socket [ 3.545953] NET: Registered protocol family 10 [ 3.548566] Segment Routing with IPv6 [ 3.549621] NET: Registered protocol family 17 [ 3.551672] mpls_gso: MPLS GSO support [ 3.555995] RAS: Correctable Errors collector initialized. [ 3.557570] AVX version of gcm_enc/dec engaged. [ 3.558654] AES CTR mode by8 optimization enabled [ 3.635498] sched_clock: Marking stable (3635461990, 0)->(4548262422, -912800432) [ 3.638600] registered taskstats version 1 [ 3.640162] Loading compiled-in X.509 certificates [ 3.641609] zswap: loaded using pool lzo/zbud [ 3.667296] Key type big_key registered [ 3.680065] Key type encrypted registered [ 3.681443] ima: No TPM chip found, activating TPM-bypass! [ 3.683174] ima: Allocated hash algorithm: sha1 [ 3.684080] ima: No architecture policies found [ 3.685174] evm: Initialising EVM extended attributes: [ 3.686376] evm: security.selinux [ 3.687065] evm: security.ima [ 3.687799] evm: security.capability [ 3.688689] evm: HMAC attrs: 0x1 [ 3.690685] rtc_cmos 00:05: setting system clock to 2026-08-27 00:56:29 UTC (1787792189) [ 3.695907] debug: unmapping init [mem 0xffffffffb3803000-0xffffffffb39fffff] [ 3.698224] debug: unmapping init [mem 0xffffffffb2582000-0xffffffffb2858fff] [ 3.707162] Write protecting the kernel read-only data: 28672k [ 3.710226] debug: unmapping init [mem 0xffffffffb0c03000-0xffffffffb0dfffff] [ 3.712375] debug: unmapping init [mem 0xffffffffb1514000-0xffffffffb15fffff] [ 3.750465] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.758499] systemd[1]: Detected virtualization kvm. [ 3.759795] systemd[1]: Detected architecture x86-64. [ 3.760984] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.793312] systemd[1]: No hostname configured. [ 3.794978] systemd[1]: Set hostname to . [ 3.796898] random: systemd: uninitialized urandom read (16 bytes read) [ 3.798519] systemd[1]: Initializing machine ID from random generator. [ 3.841863] random: ln: uninitialized urandom read (6 bytes read) [ 3.946958] random: systemd: uninitialized urandom read (16 bytes read) [ 3.950071] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.953826] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.958435] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Reached target Timers. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.573161] device-mapper: uevent: version 1.0.3 [ 4.575475] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.324069] virtio_net virtio0 ens2: renamed from eth0 [ 5.337098] random: fast init done [ 5.408950] scsi host0: ata_piix [ 5.433425] scsi host1: ata_piix [ 5.436301] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.438137] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.800388] dracut-initqueue[602]: RTNETLINK answers: File exists [ 10.195464] random: crng init done [ 10.197569] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.636811] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.852522] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.102819] SELinux: Disabled at runtime. [ 12.166594] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.176516] systemd[1]: Detected virtualization kvm. [ 12.178691] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 15.385395] systemd[1]: initrd-switch-root.service: Succeeded. [ 15.387824] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 15.393564] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 15.398275] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 15.400815] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 15.407411] systemd[1]: Starting Journal Service... Starting Journal Service... [ 15.411513] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target[ 15.456023] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Initrd File Systems. Starting Create list of required st…ce nodes for the current kernel... Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on Process Core Dump Socket. Starting Apply Kernel Variables... [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 15.860036] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 16.109860] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 16.178366] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 16.282727] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 16.311850] EDAC sbridge: Ver: 1.1.2 [ 18.627896] Key type dns_resolver registered [ 18.935194] NFS: Registering the id_resolver key type [ 18.937095] Key type id_resolver registered [ 18.938133] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg616-server login: [ 68.645537] libcfs: loading out-of-tree module taints kernel. [ 68.681984] Key type ._llcrypt registered [ 68.683151] Key type .llcrypt registered [ 68.762938] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 85.320501] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 86.127897] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 86.136041] alg: No test for adler32 (adler32-zlib) [ 87.437380] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 88.138851] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 89.847119] Key type lgssc registered [ 90.968912] Lustre: Echo OBD driver; http://www.lustre.org/ [ 105.044800] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 110.765232] hrtimer: interrupt took 2416589 ns [ 147.240083] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 157.658575] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 157.678797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 158.895295] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 158.919498] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 159.002805] Lustre: lustre-MDT0000: new disk, initializing [ 159.107273] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 159.120571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 162.790144] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 173.734944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 173.841773] Lustre: 6517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 173.870384] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 173.874978] Lustre: Skipped 1 previous similar message [ 174.001718] Lustre: lustre-MDT0001: new disk, initializing [ 174.068670] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 174.102933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 174.115913] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 177.848124] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 181.869070] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 190.139760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 190.401595] Lustre: lustre-OST0000: new disk, initializing [ 190.405639] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 190.412613] Lustre: 8454:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 190.494556] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 196.055585] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.141578] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 196.177323] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 196.298331] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 209.214395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 209.297750] Lustre: lustre-OST0001: new disk, initializing [ 209.300883] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 209.304902] Lustre: 9526:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 209.357623] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 214.518652] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 217.608198] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 217.616415] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 217.664423] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 225.087400] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 228.803604] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 234.463462] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing check_logdir /tmp/testlogs/ [ 238.408762] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing yml_node [ 242.949764] Lustre: DEBUG MARKER: Client: 2.17.57.84 [ 245.095033] Lustre: DEBUG MARKER: MDS: 2.17.57.84 [ 247.123669] Lustre: DEBUG MARKER: OSS: 2.17.57.84 [ 248.564564] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 26 21:00:33 EDT 2026 [ 263.392943] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 264.821633] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 281.567915] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 281.575494] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 281.598682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 284.128611] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 284.129412] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 284.147112] Lustre: Skipped 3 previous similar messages [ 286.416216] Lustre: server umount lustre-MDT0000 complete [ 289.445880] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792475 with bad export cookie 1256624546235126438 [ 289.451741] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 289.464834] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 294.368072] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 294.368551] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 294.370645] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 294.388544] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 294.397447] Lustre: Skipped 3 previous similar messages [ 295.759381] Lustre: server umount lustre-MDT0001 complete [ 299.276824] Lustre: server umount lustre-OST0000 complete [ 302.974439] Lustre: server umount lustre-OST0001 complete [ 309.656553] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 318.890641] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 349.785747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 349.992389] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 350.022372] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 350.095677] Lustre: lustre-MDT0000: new disk, initializing [ 350.150685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 350.167919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 354.087190] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 363.393117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 363.475562] Lustre: 15089:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 363.521340] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 363.527274] Lustre: Skipped 1 previous similar message [ 363.583451] Lustre: lustre-MDT0001: new disk, initializing [ 363.662665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 363.689639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 363.703102] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 367.599742] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 376.488530] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 381.679709] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 387.485613] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 387.780699] Lustre: lustre-OST0000: new disk, initializing [ 387.788637] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 387.797918] Lustre: 17059:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 387.891122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 389.755143] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 389.772067] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 389.881776] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 393.507587] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 402.102500] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 406.119363] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 406.410359] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 410.415989] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 410.697452] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 415.713984] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 415.725266] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 415.742837] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 415.752064] Lustre: Skipped 2 previous similar messages [ 420.839048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 420.846635] Lustre: Skipped 1 previous similar message [ 426.980111] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 427.106099] Lustre: server umount lustre-OST0000 complete [ 430.564672] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 430.569448] LustreError: Skipped 1 previous similar message [ 430.572245] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 430.581284] Lustre: Skipped 1 previous similar message [ 430.585740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 430.589571] Lustre: Skipped 2 previous similar messages [ 436.577487] Lustre: server umount lustre-MDT0000 complete [ 440.017448] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792625 with bad export cookie 1256624546235127971 [ 440.021984] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 440.032914] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 440.335195] Lustre: server umount lustre-MDT0001 complete [ 445.223875] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 21:03:50 (1787792630) [ 452.052391] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 452.423370] LustreError: 19325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 452.477453] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 456.826944] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 457.699940] LustreError: 19326:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 462.827912] LustreError: 19325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 467.247369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 472.124596] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 480.669321] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 486.210116] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 494.164760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 500.798306] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 508.692949] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 515.973784] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 523.750199] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 523.753577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 523.772128] Lustre: Skipped 3 previous similar messages [ 523.774603] Lustre: Skipped 5 previous similar messages [ 528.717499] Lustre: server umount lustre-MDT0000 complete [ 528.867111] LustreError: 19325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 528.882742] LustreError: 19325:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 531.988183] LustreError: 20061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792717 with bad export cookie 1256624546235128846 [ 531.989138] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 532.000848] LustreError: 20061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 532.248937] Lustre: server umount lustre-MDT0001 complete [ 546.039464] Lustre: server umount lustre-OST0000 complete [ 569.202328] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 578.521554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 578.910142] LustreError: 23704:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 578.982929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 578.990412] Lustre: Skipped 2 previous similar messages [ 583.250918] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 591.207643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 596.346151] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 599.371278] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 606.435293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 614.775481] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 623.121178] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 623.386701] Lustre: lustre-OST0001: new disk, initializing [ 623.394095] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 623.405126] Lustre: 26022:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 624.716698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 624.726962] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 624.867903] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 630.860386] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 639.732425] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 642.811075] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 652.770786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 652.776702] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 652.786659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 652.790533] Lustre: Skipped 1 previous similar message [ 658.327379] Lustre: server umount lustre-MDT0000 complete [ 660.969122] LustreError: 23705:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 660.988020] LustreError: 23705:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 662.063946] LustreError: 23687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792847 with bad export cookie 1256624546235129693 [ 662.082262] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 662.535524] Lustre: server umount lustre-MDT0001 complete [ 677.095748] Lustre: server umount lustre-OST0000 complete [ 691.074510] Lustre: server umount lustre-OST0001 complete [ 706.344031] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 716.746243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 717.052493] LustreError: 28991:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 717.122744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 717.131602] Lustre: Skipped 3 previous similar messages [ 721.302038] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 729.510447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 733.064908] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 736.083541] Lustre: 30139:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 743.018844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 751.487889] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 760.710952] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 767.846649] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 775.940250] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 779.633563] Lustre: 32018:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 787.951843] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 787.961848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 787.972739] Lustre: Skipped 6 previous similar messages [ 788.001530] Lustre: Skipped 7 previous similar messages [ 793.006644] Lustre: server umount lustre-MDT0000 complete [ 793.071016] LustreError: 30098:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 793.118088] LustreError: 30098:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 797.095806] LustreError: 28971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792982 with bad export cookie 1256624546235130995 [ 797.097523] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 797.102943] LustreError: 28971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 797.443917] Lustre: server umount lustre-MDT0001 complete [ 811.822872] Lustre: server umount lustre-OST0000 complete [ 813.727154] Lustre: 3652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787792983/real 1787792983] req@ffff922e3b7b9f80 x1874636070734592/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787792999 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 815.197154] Lustre: server umount lustre-OST0001 complete [ 824.690885] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 21:10:09 (1787793009) [ 853.442991] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 864.480316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 868.963314] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 878.350499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 883.638506] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 887.276708] Lustre: 35977:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 895.341728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 902.649835] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 911.528614] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 918.497373] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 926.947917] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 931.339792] Lustre: 37859:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 935.393539] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 935.406204] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 935.431451] Lustre: Skipped 1 previous similar message [ 935.436464] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 941.761494] Lustre: server umount lustre-MDT0000 complete [ 942.561503] LustreError: 34820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 942.589763] LustreError: 34820:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 945.646500] LustreError: 34800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793131 with bad export cookie 1256624546235132101 [ 945.660167] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 945.663169] LustreError: 34800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 946.024926] Lustre: server umount lustre-MDT0001 complete [ 959.652298] Lustre: server umount lustre-OST0000 complete [ 974.083369] Lustre: server umount lustre-OST0001 complete [ 983.883417] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 21:12:47 (1787793167) [ 1001.685171] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1012.035505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1012.580901] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1012.586337] Lustre: Skipped 7 previous similar messages [ 1016.271613] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1024.518636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1029.739678] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1032.898846] Lustre: 41152:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1039.810994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1046.875813] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1055.741224] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1062.545705] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1070.963531] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1075.251927] Lustre: 43029:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1076.218874] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1076.231064] Lustre: Skipped 1 previous similar message [ 1081.319994] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1081.333066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1081.347409] Lustre: Skipped 3 previous similar messages [ 1082.150945] Lustre: server umount lustre-MDT0000 complete [ 1086.279884] LustreError: 39982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793272 with bad export cookie 1256624546235133200 [ 1086.292612] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1086.297720] LustreError: 39982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1092.853416] Lustre: server umount lustre-MDT0001 complete [ 1102.080823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1111.332375] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x11706d5dee61b17e [ 1111.342787] Lustre: MGC192.168.206.116@tcp: Connection restored to 0@lo (at 0@lo) [ 1116.101558] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1120.114721] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1124.344943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1129.568806] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1129.968096] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1129.973755] Lustre: Skipped 1 previous similar message [ 1136.947598] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1138.576904] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1145.022699] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1146.560291] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1158.405600] Lustre: server umount lustre-MDT0000 complete [ 1161.855493] LustreError: 39982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793347 with bad export cookie 1256624546235134334 [ 1161.859693] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1161.862702] LustreError: 39982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1162.172250] Lustre: server umount lustre-MDT0001 complete [ 1176.656841] Lustre: server umount lustre-OST0000 complete [ 1190.288644] Lustre: server umount lustre-OST0001 complete [ 1198.922306] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 21:16:23 (1787793383) [ 1215.235253] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1226.258302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1227.026031] LustreError: 47639:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1227.048937] LustreError: 47639:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 29 previous similar messages [ 1231.954861] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1241.555111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1246.728858] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1249.837663] Lustre: 48791:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1256.152048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1262.655359] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1271.313799] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1277.485646] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1284.703754] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1289.209640] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1303.293976] Lustre: server umount lustre-MDT0000 complete [ 1303.524846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1307.108550] LustreError: 47622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793492 with bad export cookie 1256624546235135132 [ 1307.116764] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1307.124095] LustreError: 47622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1308.646557] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 1308.652088] Lustre: Skipped 16 previous similar messages [ 1309.554257] Lustre: server umount lustre-MDT0001 complete [ 1314.030941] Lustre: server umount lustre-OST0000 complete [ 1318.114735] Lustre: server umount lustre-OST0001 complete [ 1327.544470] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 21:18:32 (1787793512) [ 1330.139313] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1331.629872] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 21:18:36 (1787793516) [ 1341.292278] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 1349.115708] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1368.063638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1368.313101] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1368.411433] Lustre: lustre-MDT0000: new disk, initializing [ 1368.518490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1372.181757] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1381.450388] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1389.315042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1389.693213] Lustre: lustre-OST0000: new disk, initializing [ 1389.699593] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1389.713194] Lustre: Skipped 1 previous similar message [ 1389.728269] Lustre: 54863:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1391.285949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1391.300912] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1391.342694] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1397.048505] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1406.049764] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1408.576284] Lustre: DEBUG MARKER: create test files [ 1442.785079] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1442.798823] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1442.810748] Lustre: Skipped 15 previous similar messages [ 1445.915715] Lustre: server umount lustre-OST0000 complete [ 1450.280099] Lustre: server umount lustre-MDT0000 complete [ 1462.794239] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 1465.427202] Key type lgssc unregistered [ 1465.713335] LNet: 56495:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1465.725766] LNetError: 56495:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1465.749302] LNet: Removed LNI 192.168.206.116@tcp [ 1466.734143] Key type .llcrypt unregistered [ 1466.740637] Key type ._llcrypt unregistered [ 1492.138473] Key type ._llcrypt registered [ 1492.144779] Key type .llcrypt registered [ 1492.241288] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1493.217418] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1493.260186] alg: No test for adler32 (adler32-zlib) [ 1494.235869] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 1494.433479] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 1496.071168] Key type lgssc registered [ 1496.984681] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1504.540792] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1504.560478] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1506.101019] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1510.290355] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1517.022841] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1523.516807] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1523.777794] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1529.533681] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1530.925947] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1531.071904] Lustre: 59199:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1531.081567] Lustre: 59199:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1531.088891] Lustre: 59199:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1531.095649] Lustre: 59199:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1531.100101] Lustre: 59199:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1531.104768] Lustre: 59199:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1536.409402] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1541.187906] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1546.743211] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1555.347435] Lustre: server umount lustre-OST0000 complete [ 1568.870214] Lustre: server umount lustre-MDT0000 complete [ 1580.469304] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 1583.546038] Key type lgssc unregistered [ 1583.854479] LNet: 60807:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1583.863291] LNetError: 60807:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1584.938523] LNet: Removed LNI 192.168.206.116@tcp [ 1585.896307] Key type .llcrypt unregistered [ 1585.898935] Key type ._llcrypt unregistered [ 1616.492642] Key type ._llcrypt registered [ 1616.496189] Key type .llcrypt registered [ 1616.576122] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1617.640000] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1617.720056] alg: No test for adler32 (adler32-zlib) [ 1618.803995] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 1619.024173] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 1620.671950] Key type lgssc registered [ 1621.717684] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1628.716545] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1628.737562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1630.167698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1634.353675] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1641.685419] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1647.878410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1648.194672] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1654.284549] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1654.381410] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1654.513609] Lustre: 63639:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1654.527476] Lustre: 63639:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1654.532128] Lustre: 63639:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1654.540121] Lustre: 63639:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1654.547163] Lustre: 63639:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1654.559403] Lustre: 63639:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1662.228870] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1668.470864] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1680.357775] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1680.376141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1680.390373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1683.794290] Lustre: server umount lustre-OST0000 complete [ 1687.797747] Lustre: server umount lustre-MDT0000 complete [ 1698.779851] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 1701.177932] Key type lgssc unregistered [ 1701.425742] LNet: 65197:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1701.436707] LNetError: 65197:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1701.456135] LNet: Removed LNI 192.168.206.116@tcp [ 1702.154596] Key type .llcrypt unregistered [ 1702.160311] Key type ._llcrypt unregistered [ 1720.255910] Key type ._llcrypt registered [ 1720.258403] Key type .llcrypt registered [ 1720.370657] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 1736.594270] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1737.685313] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1737.763890] alg: No test for adler32 (adler32-zlib) [ 1738.860996] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 1739.203776] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 1740.943178] Key type lgssc registered [ 1742.088931] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1779.781244] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1779.838561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1781.119208] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1781.163725] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1781.322586] Lustre: lustre-MDT0000: new disk, initializing [ 1781.427286] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1781.447507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1786.003286] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1798.993927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1799.199048] Lustre: 68347:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1799.321658] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1799.325293] Lustre: Skipped 1 previous similar message [ 1799.396906] Lustre: lustre-MDT0001: new disk, initializing [ 1799.567393] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1799.626229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1799.637378] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1804.965205] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1815.820824] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1822.211601] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1830.637512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1830.992067] Lustre: lustre-OST0000: new disk, initializing [ 1830.995166] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1831.006319] Lustre: 70321:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1831.091493] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1837.408663] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1838.159613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1838.170274] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1838.342986] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1847.376123] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1854.522923] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1854.878079] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1860.304900] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1860.515806] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1864.180459] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1864.189140] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1864.221758] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1864.230821] Lustre: Skipped 1 previous similar message [ 1869.183814] Lustre: server umount lustre-OST0000 complete [ 1874.403151] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1874.405057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1874.418532] Lustre: Skipped 2 previous similar messages [ 1879.330304] Lustre: server umount lustre-MDT0000 complete [ 1883.677549] LustreError: 68339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794069 with bad export cookie 6507296814356571623 [ 1883.691333] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1883.693469] LustreError: 68339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1883.974211] Lustre: server umount lustre-MDT0001 complete [ 1892.758977] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 21:27:57 (1787794077) [ 1903.408399] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1948.588121] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 1958.991364] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1994.514570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1994.759132] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1994.801503] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1994.959915] Lustre: lustre-MDT0000: new disk, initializing [ 1995.088820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1995.119561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1999.374538] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2010.255427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2010.547557] Lustre: 76341:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2010.602314] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2010.605881] Lustre: Skipped 1 previous similar message [ 2010.697357] Lustre: lustre-MDT0001: new disk, initializing [ 2010.818292] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2010.847569] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2010.869270] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2014.908588] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2024.313183] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2030.320357] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2037.245911] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2037.639406] Lustre: lustre-OST0000: new disk, initializing [ 2037.646733] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2037.655816] Lustre: 78308:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2037.762491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2044.281544] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2044.522401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2044.538343] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2044.734307] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2054.456817] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2060.079043] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2060.392761] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2065.592571] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2065.811664] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2070.502307] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2070.510664] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2070.512686] LustreError: Skipped 1 previous similar message [ 2070.513861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2070.513866] Lustre: Skipped 1 previous similar message [ 2070.556515] Lustre: Skipped 1 previous similar message [ 2075.621918] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2075.626460] Lustre: Skipped 2 previous similar messages [ 2080.743256] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2080.750440] Lustre: Skipped 1 previous similar message [ 2082.271290] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2082.475695] Lustre: server umount lustre-OST0000 complete [ 2087.904684] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2087.913623] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2090.982918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2090.990340] Lustre: Skipped 1 previous similar message [ 2092.616036] Lustre: server umount lustre-MDT0000 complete [ 2096.103633] LustreError: 76352:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2096.120425] LustreError: 76352:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2096.409996] LustreError: 76334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794282 with bad export cookie 6507296814356572498 [ 2096.414872] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2096.427355] LustreError: 76334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2096.752611] Lustre: server umount lustre-MDT0001 complete [ 2105.687288] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 21:31:30 (1787794290) [ 2112.099996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2112.489616] LustreError: 80726:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2112.561616] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2116.574904] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2117.601233] LustreError: 80727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2122.722281] LustreError: 80726:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2124.408133] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2129.050561] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2136.426722] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2142.311935] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2149.283351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2149.596844] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2149.607099] Lustre: Skipped 1 previous similar message [ 2156.811450] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2165.574510] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2170.308489] Lustre: 83399:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting cc483f4e-322e-4057-9edd-1bd4fab9d168 at adminstrative request [ 2175.461856] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2175.481690] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2185.707417] Lustre: MGS: Client cc483f4e-322e-4057-9edd-1bd4fab9d168 (at 0@lo) reconnecting [ 2185.722265] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x5a4e8ff9bbc12ccb to 0x5a4e8ff9bbc12f6b [ 2185.780614] Lustre: MGC192.168.206.116@tcp: Connection restored to 0@lo (at 0@lo) [ 2207.585054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2207.748972] Lustre: lustre-OST0001: new disk, initializing [ 2207.751845] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2207.755950] Lustre: 83791:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2214.263567] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2218.092088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2218.098726] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2218.176897] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2223.888762] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2228.708330] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2228.718585] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2228.720714] LustreError: Skipped 1 previous similar message [ 2228.743955] Lustre: Skipped 1 previous similar message [ 2228.745878] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2232.893411] Lustre: server umount lustre-OST0001 complete [ 2233.838606] LustreError: 82550:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2233.857675] LustreError: 82550:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2236.896219] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2236.909231] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2236.925745] Lustre: Skipped 1 previous similar message [ 2242.346593] Lustre: server umount lustre-OST0000 complete [ 2247.652203] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2252.284512] Lustre: server umount lustre-MDT0000 complete [ 2254.313754] LustreError: 80727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2254.331652] LustreError: 80727:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2256.040350] LustreError: 81462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794441 with bad export cookie 6507296814356574059 [ 2256.051321] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2256.312929] Lustre: server umount lustre-MDT0001 complete [ 2265.358219] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 21:34:09 (1787794449) [ 2267.086940] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2269.193917] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 21:34:13 (1787794453) [ 2270.671459] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2272.468413] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 21:34:16 (1787794456) [ 2274.183402] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2276.051637] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 21:34:20 (1787794460) [ 2277.621326] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2279.703762] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 21:34:23 (1787794463) [ 2281.207511] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2287.370656] loop: module loaded [ 2287.403425] loop0: detected capacity change from 0 to 4294967296 [ 2287.460445] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2289.336488] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2293.170144] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2297.057064] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2326.454754] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2326.511944] LustreError: 87917:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2326.517233] LustreError: 87917:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2326.523649] LustreError: 87917:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2326.533481] LustreError: 87917:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2346.792597] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 2358.024766] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 2392.195636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2392.450842] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2392.470768] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2392.546490] Lustre: lustre-MDT0000: new disk, initializing [ 2392.600464] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2392.603700] Lustre: Skipped 1 previous similar message [ 2392.614573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2396.250653] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2404.906683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2405.025292] Lustre: 90783:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2405.246712] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2409.966436] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2418.566712] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2423.492702] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2429.299172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2429.564682] Lustre: 92753:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2435.680962] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2435.825536] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2436.201281] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2444.564951] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2448.688907] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2448.886744] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2453.692325] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2453.931597] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2456.563366] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2456.564381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2456.579176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2456.592446] Lustre: Skipped 4 previous similar messages [ 2456.606548] Lustre: Skipped 8 previous similar messages [ 2461.669328] LustreError: 92733:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2461.680056] LustreError: 92733:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2461.787121] Lustre: server umount lustre-OST0000 complete [ 2471.365096] Lustre: server umount lustre-MDT0000 complete [ 2474.421226] LustreError: 90774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794660 with bad export cookie 6507296814356574675 [ 2474.427578] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2474.432433] LustreError: 90774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2474.697922] Lustre: server umount lustre-MDT0001 complete [ 2482.196634] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 21:37:46 (1787794666) [ 2488.281330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2492.731879] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2493.921629] LustreError: 95167:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2493.939516] LustreError: 95167:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2501.315127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2505.932194] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2513.055666] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2519.400882] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2526.358903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2526.670530] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2526.675794] Lustre: Skipped 4 previous similar messages [ 2533.305477] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2540.483749] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2544.946343] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2545.127082] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2549.409125] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2549.675170] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2556.612197] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2556.807695] Lustre: lustre-OST0001: new disk, initializing [ 2556.811941] Lustre: Skipped 2 previous similar messages [ 2556.817082] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2556.824106] Lustre: Skipped 3 previous similar messages [ 2556.833454] Lustre: 98537:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2562.148640] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2567.240913] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2567.250038] Lustre: Skipped 2 previous similar messages [ 2567.262381] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2567.361575] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2571.082745] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2575.872510] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2576.090946] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2579.696827] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2579.864240] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2597.196661] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2621.096894] Lustre: Failing over lustre-MDT0000 [ 2621.625365] Lustre: server umount lustre-MDT0000 complete [ 2622.783071] LustreError: 99791:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2622.800687] LustreError: 99791:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2623.969777] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2623.991409] Lustre: Skipped 4 previous similar messages [ 2639.329202] Lustre: 66570:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787794809/real 1787794809] req@00000000fb2df958 x1874637802971008/t0(0) o400->MGC192.168.206.116@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787794825 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2639.369632] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2645.593558] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2645.596858] LDISKFS-fs (dm-0): recovery complete [ 2645.618563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2649.573915] LustreError: 66566:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000006d3dd179 x1874637802979072/t0(0) o250->MGC192.168.206.116@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2649.904275] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2653.494860] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2653.497979] Lustre: 95162:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2654.101280] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2655.206893] Lustre: 95167:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2655.224303] Lustre: 95167:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2655.266132] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2655.348681] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2655.664906] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2656.002243] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2656.012232] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2656.335139] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2656.342338] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2656.353505] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2656.366125] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2657.016063] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2657.026788] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2657.343166] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2657.357949] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2658.643651] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2658.657934] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2659.306389] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2659.318558] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2659.648414] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2659.661206] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2662.833551] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2662.846276] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2663.472299] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2663.482224] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2663.807118] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2663.820538] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2671.097113] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2671.106156] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2671.761810] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2671.771415] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2672.087620] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2672.099037] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2687.284629] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2687.294765] Lustre: 101383:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2687.937604] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2687.955853] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2688.279126] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2688.286458] LustreError: 101383:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2713.271160] Lustre: 101383:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2713.285573] Lustre: 101383:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2713.301170] Lustre: 101383:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 02ed206c-7449-4eb2-8e2e-4de30f59ec79@192.168.206.16@tcp [ 2713.323367] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2713.332537] Lustre: 101383:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000c60a2d53 x1874637785993984/t0(8589934779) o101->02ed206c-7449-4eb2-8e2e-4de30f59ec79@192.168.206.16@tcp:209/0 lens 592/0 e 0 to 0 dl 1787794909 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2713.375373] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2713.376835] Lustre: lustre-MDT0000: Denying connection for new client 02ed206c-7449-4eb2-8e2e-4de30f59ec79 (at 192.168.206.16@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2713.407226] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2713.458377] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2713.461629] Lustre: Skipped 2 previous similar messages [ 2713.467174] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2713.492944] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2713.501253] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2726.894816] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2726.903111] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2726.910021] Lustre: Skipped 2 previous similar messages [ 2726.918323] Lustre: Skipped 3 previous similar messages [ 2732.713857] Lustre: server umount lustre-OST0000 complete [ 2736.611896] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2736.617808] LustreError: Skipped 1 previous similar message [ 2742.199403] Lustre: server umount lustre-OST0001 complete [ 2760.159657] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2760.423410] Lustre: server umount lustre-MDT0000 complete [ 2762.729094] LustreError: 96833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2762.745192] LustreError: 96833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 37 previous similar messages [ 2763.903203] LustreError: 98535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794949 with bad export cookie 6507296814356625894 [ 2763.910709] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2763.915187] LustreError: 98535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2764.465874] Lustre: server umount lustre-MDT0001 complete [ 2772.262462] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 21:42:36 (1787794956) [ 2779.301668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2779.305602] Lustre: *** cfs_fail_loc=197, val=0*** [ 2794.847587] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2799.968215] LustreError: 103356:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.116@tcp: failed processing log, type 4: rc = -110 [ 2805.115714] Lustre: *** cfs_fail_loc=197, val=0*** [ 2805.119719] Lustre: Skipped 65 previous similar messages [ 2825.696039] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2825.704214] Lustre: Skipped 2 previous similar messages [ 2831.698659] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2838.374773] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2840.279516] Lustre: server umount lustre-OST0000 complete [ 2848.133282] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 21:43:52 (1787795032) [ 2869.971766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2874.076646] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2882.306584] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2885.520096] LustreError: 105007:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2885.527255] LustreError: 105007:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2885.531954] LustreError: 105007:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 15, retries 0, failed: rc = -5 [ 2885.794770] Lustre: server umount lustre-MDT0000 complete [ 2915.793965] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 2925.666396] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 2960.738655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2961.010992] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2961.080765] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2961.220840] Lustre: lustre-MDT0000: new disk, initializing [ 2961.363753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2965.426614] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2974.993086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2975.146184] Lustre: 109180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2975.430659] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2979.558773] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2989.038287] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2994.655695] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3001.117688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3001.401383] Lustre: 111149:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3007.548076] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3009.240879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3017.787970] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3022.571538] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3022.791655] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3026.642394] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3026.845496] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3029.995940] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3029.998954] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3030.006074] LustreError: Skipped 1 previous similar message [ 3030.031032] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3030.040240] Lustre: Skipped 5 previous similar messages [ 3030.062462] Lustre: Skipped 12 previous similar messages [ 3034.874811] Lustre: server umount lustre-OST0000 complete [ 3044.858864] Lustre: server umount lustre-MDT0000 complete [ 3048.416157] LustreError: 109191:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3048.433386] LustreError: 109191:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3048.839667] LustreError: 109171:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795234 with bad export cookie 6507296814356636569 [ 3048.841238] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3048.854428] LustreError: 109171:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3049.199891] Lustre: server umount lustre-MDT0001 complete [ 3058.360091] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 21:47:22 (1787795242) [ 3067.125788] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3069.809489] Key type lgssc unregistered [ 3070.123721] LNet: 113635:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3070.131136] LNetError: 113635:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3070.151464] LNet: Removed LNI 192.168.206.116@tcp [ 3071.094317] Key type .llcrypt unregistered [ 3071.100406] Key type ._llcrypt unregistered [ 3089.237941] Key type ._llcrypt registered [ 3089.242646] Key type .llcrypt registered [ 3089.368290] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 3103.480577] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3104.631188] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3104.665215] alg: No test for adler32 (adler32-zlib) [ 3105.759737] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 3106.049474] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3107.767153] Key type lgssc registered [ 3109.114070] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3146.172172] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3146.188738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3147.391649] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3147.417272] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3147.487182] Lustre: lustre-MDT0000: new disk, initializing [ 3147.550239] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3147.569455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3150.886492] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3158.678287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3158.866700] Lustre: lustre-OST0000: new disk, initializing [ 3158.874744] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3158.883251] Lustre: Skipped 1 previous similar message [ 3158.890223] Lustre: 117805:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3158.982773] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3163.148911] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3165.203399] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3165.210769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3165.228721] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3169.813073] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3169.923466] Lustre: lustre-OST0001: new disk, initializing [ 3169.928431] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3169.935951] Lustre: 118669:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3170.001521] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3173.845750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3173.852365] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3173.879128] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3174.393924] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3184.101515] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3184.115877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3184.120569] Lustre: Skipped 1 previous similar message [ 3187.099713] Lustre: server umount lustre-MDT0000 complete [ 3192.147060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3196.234798] LustreError: 116993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795382 with bad export cookie 5377919986535403049 [ 3196.235667] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3196.244940] LustreError: 116993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3196.327107] Lustre: server umount lustre-OST0000 complete [ 3199.970943] Lustre: server umount lustre-OST0001 complete [ 3208.194969] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 21:49:52 (1787795392) [ 3226.344243] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 21:50:11 (1787795411) [ 3234.060226] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3252.932317] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 21:50:37 (1787795437) [ 3286.275612] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 3295.126706] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3331.064089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3331.395784] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3331.432619] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3331.541827] Lustre: lustre-MDT0000: new disk, initializing [ 3331.611771] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3331.633802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3336.140488] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3346.825728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3346.937559] Lustre: 126435:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3346.991926] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3346.996363] Lustre: Skipped 1 previous similar message [ 3347.080255] Lustre: lustre-MDT0001: new disk, initializing [ 3347.195776] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3347.240275] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3347.247925] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3350.505456] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3359.607938] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3366.617756] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3375.338459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3375.558286] Lustre: lustre-OST0000: new disk, initializing [ 3375.568142] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3375.578376] Lustre: 128405:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3375.690626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3382.304592] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3384.420480] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3384.437738] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3384.539667] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3392.514150] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3396.925460] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3397.144342] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3401.548776] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3401.796505] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3405.281825] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3405.285014] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3405.290078] LustreError: Skipped 1 previous similar message [ 3405.290687] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3405.321470] Lustre: Skipped 2 previous similar messages [ 3410.281212] Lustre: server umount lustre-OST0000 complete [ 3413.987619] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3413.994077] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3414.005497] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3414.011563] Lustre: Skipped 1 previous similar message [ 3420.640495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3420.648483] Lustre: Skipped 3 previous similar messages [ 3425.764644] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3425.777487] Lustre: Skipped 1 previous similar message [ 3428.320141] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3428.508085] Lustre: server umount lustre-MDT0000 complete [ 3432.411279] LustreError: 126428:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795618 with bad export cookie 5377919986535404274 [ 3432.419561] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3432.819500] Lustre: server umount lustre-MDT0001 complete [ 3441.041621] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 21:53:45 (1787795625) [ 3447.469644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3448.042389] LustreError: 130819:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3448.058837] LustreError: 130819:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3448.106044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3452.770341] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3453.409074] LustreError: 130820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3457.506836] LustreError: 130819:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3460.758858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3464.959613] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3471.734219] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3477.019736] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3483.060600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3483.535976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3483.547859] Lustre: Skipped 1 previous similar message [ 3489.299116] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3496.638967] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3500.657917] Lustre: *** cfs_fail_loc=159, val=0*** [ 3516.232239] Lustre: lustre-MDT0000: Client cc5b6a54-6ee3-444f-98f9-04102b6a9b8f (at 192.168.206.16@tcp) reconnecting [ 3517.767856] Lustre: *** cfs_fail_loc=159, val=0*** [ 3517.778610] Lustre: Skipped 5 previous similar messages [ 3533.131436] Lustre: lustre-MDT0000: Client cc5b6a54-6ee3-444f-98f9-04102b6a9b8f (at 192.168.206.16@tcp) reconnecting [ 3538.917987] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3538.928253] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3538.930197] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3538.938634] Lustre: Skipped 2 previous similar messages [ 3538.957417] Lustre: Skipped 1 previous similar message [ 3539.865463] Lustre: server umount lustre-OST0000 complete [ 3544.045882] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3544.057805] Lustre: Skipped 1 previous similar message [ 3549.926561] Lustre: server umount lustre-MDT0000 complete [ 3553.493265] LustreError: 130799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795739 with bad export cookie 5377919986535405156 [ 3553.500735] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3553.506065] LustreError: 130799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3553.826971] Lustre: server umount lustre-MDT0001 complete [ 3563.260890] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3566.026468] Key type lgssc unregistered [ 3566.244132] LNet: 134592:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3566.251862] LNetError: 134592:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3566.270644] LNet: Removed LNI 192.168.206.116@tcp [ 3567.134882] Key type .llcrypt unregistered [ 3567.140575] Key type ._llcrypt unregistered [ 3590.427855] Key type ._llcrypt registered [ 3590.430673] Key type .llcrypt registered [ 3590.554092] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3591.397603] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3591.486422] alg: No test for adler32 (adler32-zlib) [ 3592.510376] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 3592.771274] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3594.482692] Key type lgssc registered [ 3595.507736] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3602.554504] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3602.567251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3603.891390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3607.555392] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3614.178300] LustreError: 136005:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3614.201164] LustreError: 136005:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3615.334482] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3615.687996] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3620.014784] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3626.709915] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3632.183778] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3638.603264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3638.880861] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3645.359372] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3649.143197] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3652.558822] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3662.332608] Lustre: *** cfs_fail_loc=159, val=0*** [ 3678.559233] Lustre: lustre-MDT0000: Client abb64e64-3662-4d5d-b918-21323c4c531e (at 192.168.206.16@tcp) reconnecting [ 3680.839511] Lustre: *** cfs_fail_loc=159, val=0*** [ 3696.467867] Lustre: lustre-MDT0001: Client abb64e64-3662-4d5d-b918-21323c4c531e (at 192.168.206.16@tcp) reconnecting [ 3697.781637] Lustre: *** cfs_fail_loc=159, val=0*** [ 3697.789233] Lustre: Skipped 5 previous similar messages [ 3713.341414] Lustre: lustre-MDT0001: Client abb64e64-3662-4d5d-b918-21323c4c531e (at 192.168.206.16@tcp) reconnecting [ 3717.121443] Lustre: 139041:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3719.831374] Lustre: *** cfs_fail_loc=159, val=0*** [ 3719.839754] Lustre: Skipped 2 previous similar messages [ 3735.369782] Lustre: lustre-MDT0000: Client dae4f8aa-d30b-4cdf-9ff1-76bcd4606aa8 (at 192.168.206.16@tcp) reconnecting [ 3736.946284] Lustre: *** cfs_fail_loc=159, val=0*** [ 3736.956034] Lustre: Skipped 13 previous similar messages [ 3752.272514] Lustre: lustre-MDT0000: Client dae4f8aa-d30b-4cdf-9ff1-76bcd4606aa8 (at 192.168.206.16@tcp) reconnecting [ 3754.112838] Lustre: 139335:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3759.075738] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3759.093466] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3759.104890] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3762.146251] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3762.146778] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3762.150029] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3765.045501] Lustre: server umount lustre-OST0000 complete [ 3769.312058] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3769.320189] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3769.344499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3769.352764] Lustre: Skipped 1 previous similar message [ 3772.390352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3772.391397] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3774.687472] Lustre: server umount lustre-MDT0000 complete [ 3777.504749] LustreError: 136006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3777.530689] LustreError: 136006:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3777.773088] LustreError: 135986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795963 with bad export cookie 17831043700193886275 [ 3777.780949] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3777.787080] LustreError: 135986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3778.123575] Lustre: server umount lustre-MDT0001 complete [ 3787.957751] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3791.246913] Key type lgssc unregistered [ 3791.570102] LNet: 140387:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3791.581206] LNetError: 140387:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3791.595471] LNet: Removed LNI 192.168.206.116@tcp [ 3792.662913] Key type .llcrypt unregistered [ 3792.668278] Key type ._llcrypt unregistered [ 3818.430505] Key type ._llcrypt registered [ 3818.435289] Key type .llcrypt registered [ 3818.559476] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3819.451804] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3819.518340] alg: No test for adler32 (adler32-zlib) [ 3820.781187] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 3821.105281] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3822.831556] Key type lgssc registered [ 3824.431267] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3832.099737] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3832.131106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3833.553586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3837.421350] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3844.069781] LustreError: 141799:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3844.085165] LustreError: 141799:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3845.105935] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3845.485289] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3850.087669] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3858.148417] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3863.709065] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3869.888403] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3870.218257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3871.310399] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3876.525878] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3878.457535] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3883.418937] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3889.830928] Lustre: 144393:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3892.754909] Lustre: 144442:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3894.205801] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3895.029620] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3898.317761] Lustre: 144588:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3899.143662] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3900.151931] Lustre: 144687:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3903.969717] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3903.976971] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3903.992053] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3903.996605] Lustre: Skipped 1 previous similar message [ 3904.004324] Lustre: Skipped 1 previous similar message [ 3909.089178] LustreError: 143697:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3909.109714] LustreError: 143697:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3909.202397] Lustre: server umount lustre-OST0000 complete [ 3914.212654] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3914.214458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3914.230890] Lustre: Skipped 1 previous similar message [ 3914.238220] Lustre: Skipped 1 previous similar message [ 3918.751923] Lustre: server umount lustre-MDT0000 complete [ 3919.330319] LustreError: 141799:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3919.353429] LustreError: 141799:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3921.864620] LustreError: 141780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796107 with bad export cookie 3711336927597967466 [ 3921.865470] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3921.875666] LustreError: 141780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3922.117717] Lustre: server umount lustre-MDT0001 complete [ 3930.544671] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3932.778492] Key type lgssc unregistered [ 3933.051438] LNet: 145743:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3933.055231] LNetError: 145743:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3933.064568] LNet: Removed LNI 192.168.206.116@tcp [ 3933.795165] Key type .llcrypt unregistered [ 3933.797393] Key type ._llcrypt unregistered [ 3955.624738] Key type ._llcrypt registered [ 3955.630926] Key type .llcrypt registered [ 3955.730572] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3956.626684] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3956.665891] alg: No test for adler32 (adler32-zlib) [ 3957.729622] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 3957.984982] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3959.671191] Key type lgssc registered [ 3960.677118] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3968.189265] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3968.203124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3969.636686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3975.813725] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3979.745864] LustreError: 147154:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3979.788903] LustreError: 147154:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3984.866815] LustreError: 147155:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3987.387671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3988.001127] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3994.276466] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4002.103525] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4008.010642] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4015.435208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4015.911730] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4021.681824] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4029.616869] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4032.315282] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4039.478479] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4046.212248] Lustre: *** cfs_fail_loc=159, val=0*** [ 4062.551321] Lustre: lustre-MDT0000: Client acbceaf1-67ba-4dab-8c19-0c3dc736baf1 (at 192.168.206.16@tcp) reconnecting [ 4067.821864] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4067.829474] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4067.840976] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4068.322227] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4068.331893] LustreError: Skipped 7 previous similar messages [ 4068.335549] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4068.348131] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4070.370409] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4070.378172] Lustre: Skipped 1 previous similar message [ 4075.498820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4075.511090] Lustre: Skipped 1 previous similar message [ 4080.611925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4080.616331] Lustre: Skipped 1 previous similar message [ 4082.143143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4082.323615] Lustre: server umount lustre-OST0000 complete [ 4090.341352] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4090.350142] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4090.362839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4092.381260] Lustre: server umount lustre-MDT0000 complete [ 4095.560400] LustreError: 147134:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796281 with bad export cookie 7135438479938938337 [ 4095.566620] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4095.572738] LustreError: 147134:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4095.895171] Lustre: server umount lustre-MDT0001 complete [ 4106.122541] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4108.866514] Key type lgssc unregistered [ 4109.136722] LNet: 150846:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4109.147829] LNetError: 150846:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4109.172274] LNet: Removed LNI 192.168.206.116@tcp [ 4110.105316] Key type .llcrypt unregistered [ 4110.107845] Key type ._llcrypt unregistered [ 4133.136410] Key type ._llcrypt registered [ 4133.139560] Key type .llcrypt registered [ 4133.224457] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4133.805381] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4133.832213] alg: No test for adler32 (adler32-zlib) [ 4134.811654] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 4135.080109] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4136.735960] Key type lgssc registered [ 4138.139491] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4146.000705] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4146.025600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4147.476959] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4153.625593] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4157.922220] LustreError: 152257:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4157.969880] LustreError: 152257:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4163.042699] LustreError: 152258:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4163.493103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4164.051889] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4168.276549] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4175.278166] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4181.111114] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4188.476434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4188.586685] LustreError: 154084:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4188.941559] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4193.087307] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4193.093375] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4195.272641] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4202.515406] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4221.203155] Lustre: lustre-MDT0001: evicting a9f53fcd-556f-4b03-883f-fe24cac8311f (at 192.168.206.16@tcp) by administrative request [ 4221.210641] Lustre: Skipped 1 previous similar message [ 4254.689769] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4254.697323] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4254.698473] LustreError: Skipped 1 previous similar message [ 4254.705360] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4254.720060] Lustre: Skipped 1 previous similar message [ 4254.757587] Lustre: Skipped 1 previous similar message [ 4256.412450] Lustre: server umount lustre-OST0000 complete [ 4259.810331] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4259.822060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4259.827726] Lustre: Skipped 1 previous similar message [ 4264.928436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4264.937095] Lustre: Skipped 1 previous similar message [ 4265.769796] Lustre: server umount lustre-MDT0000 complete [ 4269.925781] LustreError: 152239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796455 with bad export cookie 7194314591506078870 [ 4269.937397] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4269.946146] LustreError: 152239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4270.315711] Lustre: server umount lustre-MDT0001 complete [ 4281.431153] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4284.017972] Key type lgssc unregistered [ 4284.262338] LNet: 156329:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4284.269293] LNetError: 156329:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4284.299929] LNet: Removed LNI 192.168.206.116@tcp [ 4285.110230] Key type .llcrypt unregistered [ 4285.112853] Key type ._llcrypt unregistered [ 4360.928421] Key type ._llcrypt registered [ 4360.930132] Key type .llcrypt registered [ 4361.089291] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4362.748824] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4362.814566] alg: No test for adler32 (adler32-zlib) [ 4364.014778] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 4364.274632] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4366.055405] Key type lgssc registered [ 4367.653143] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4375.095183] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4375.119267] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4376.485337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4381.859232] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4386.793391] LustreError: 158812:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4386.823116] LustreError: 158812:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4390.886174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4391.319525] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4396.213941] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4404.580445] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4410.028527] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4416.469180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4416.921044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4418.009588] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4422.170918] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4422.937270] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4431.029890] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4442.597321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4442.623709] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4447.653847] Lustre: server umount lustre-OST0000 complete [ 4452.835117] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4452.839536] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4452.855721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4452.881069] Lustre: Skipped 2 previous similar messages [ 4452.888568] Lustre: Skipped 1 previous similar message [ 4457.966618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4457.976178] Lustre: Skipped 2 previous similar messages [ 4458.257488] Lustre: server umount lustre-MDT0000 complete [ 4462.162178] LustreError: 158793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796647 with bad export cookie 15082583207972299177 [ 4462.163641] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4462.170048] LustreError: 158793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4462.550790] Lustre: server umount lustre-MDT0001 complete [ 4473.388451] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4476.822710] Key type lgssc unregistered [ 4477.193528] LNet: 162350:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4477.204720] LNetError: 162350:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4477.225446] LNet: Removed LNI 192.168.206.116@tcp [ 4478.365543] Key type .llcrypt unregistered [ 4478.370811] Key type ._llcrypt unregistered [ 4503.139584] Key type ._llcrypt registered [ 4503.152713] Key type .llcrypt registered [ 4503.265333] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4504.314076] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4504.415550] alg: No test for adler32 (adler32-zlib) [ 4505.559735] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 4505.818877] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4507.480145] Key type lgssc registered [ 4509.005605] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4524.675346] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:11:49 (1787796709) [ 4542.509349] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 4553.656295] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4589.517589] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4589.583891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4590.931295] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4590.962888] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4591.074558] Lustre: lustre-MDT0000: new disk, initializing [ 4591.226482] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4591.273788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4595.686373] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4606.156185] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4606.265716] Lustre: 166613:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4606.306763] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4606.311829] Lustre: Skipped 1 previous similar message [ 4606.396929] Lustre: lustre-MDT0001: new disk, initializing [ 4606.546994] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4606.609621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4606.643587] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4611.122067] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4620.700534] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4626.787846] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4634.364653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4634.685183] Lustre: lustre-OST0000: new disk, initializing [ 4634.689681] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4634.695880] Lustre: 168580:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4634.774493] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4640.312868] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4640.321736] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4640.439500] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4641.236595] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4650.532711] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4661.227566] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4661.229411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4661.235298] LustreError: Skipped 1 previous similar message [ 4661.236355] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4661.260228] Lustre: Skipped 1 previous similar message [ 4665.326341] Lustre: server umount lustre-OST0000 complete [ 4671.464610] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4671.476353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4671.484175] Lustre: Skipped 2 previous similar messages [ 4676.581587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4676.589584] Lustre: Skipped 1 previous similar message [ 4681.697671] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4681.708589] Lustre: Skipped 1 previous similar message [ 4683.231124] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4683.428589] Lustre: server umount lustre-MDT0000 complete [ 4686.993507] LustreError: 166604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796872 with bad export cookie 3852132639380352306 [ 4686.999121] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4687.003364] LustreError: 166604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4687.378494] Lustre: server umount lustre-MDT0001 complete [ 4698.583956] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4701.467549] Key type lgssc unregistered [ 4701.849637] LNet: 170485:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4701.862539] LNetError: 170485:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4701.886720] LNet: Removed LNI 192.168.206.116@tcp [ 4702.851364] Key type .llcrypt unregistered [ 4702.855470] Key type ._llcrypt unregistered [ 4727.186246] Key type ._llcrypt registered [ 4727.188407] Key type .llcrypt registered [ 4727.311967] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4728.863794] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4728.929722] alg: No test for adler32 (adler32-zlib) [ 4730.041920] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 4730.331786] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4732.039408] Key type lgssc registered [ 4733.064970] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4741.003245] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4741.022602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4742.561664] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4747.439554] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4752.873235] LustreError: 171896:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4752.899591] LustreError: 171896:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4756.489695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4756.904390] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4761.408258] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4769.636554] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4775.479287] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4782.502389] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4782.910565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4788.021069] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4788.784799] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4796.787134] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4807.536920] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4807.568645] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4807.580847] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4807.595714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.616221] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4808.631619] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4808.641950] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4808.649790] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4808.656867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.672797] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4809.674091] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4809.683159] LustreError: 171916:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4809.696556] LustreError: 171916:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4809.698637] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4809.733325] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4809.738962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.750115] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4810.481615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.265987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.205426] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4812.211258] Lustre: Skipped 2 previous similar messages [ 4812.232658] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4812.243540] Lustre: Skipped 2 previous similar messages [ 4812.248052] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4812.255460] Lustre: Skipped 2 previous similar messages [ 4812.261872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.276692] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4812.286429] Lustre: Skipped 2 previous similar messages [ 4813.166697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.183176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.174535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.171579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.290695] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4817.305041] Lustre: Skipped 4 previous similar messages [ 4817.321988] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4817.333187] Lustre: Skipped 4 previous similar messages [ 4817.349484] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4817.354481] Lustre: Skipped 4 previous similar messages [ 4817.360747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.382271] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4817.389854] Lustre: Skipped 4 previous similar messages [ 4818.275608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.139492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.157247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.091241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.922983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.813449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.552923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.359786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.115634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.077057] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4826.083501] Lustre: Skipped 9 previous similar messages [ 4826.097455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4826.107700] Lustre: Skipped 9 previous similar messages [ 4826.118831] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4826.126669] Lustre: Skipped 9 previous similar messages [ 4826.133807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.146193] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4826.149842] Lustre: Skipped 9 previous similar messages [ 4827.088698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.999557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.829957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.757640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.718373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.668135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.634525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.588408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.472940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.533474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.558527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.542159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.403822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.407442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.553417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.403983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.417316] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4842.423198] Lustre: Skipped 16 previous similar messages [ 4842.442823] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4842.453546] Lustre: Skipped 16 previous similar messages [ 4842.461161] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4842.472737] Lustre: Skipped 16 previous similar messages [ 4842.477477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.487496] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4842.493801] Lustre: Skipped 16 previous similar messages [ 4843.445626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.580771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.411978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.560233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.546949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.624964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.689525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.539110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.406447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.500810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.443252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.394874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.620308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.699373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.820254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.847262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.013744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.974523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.966817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.046279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.117046] LustreError: 171916:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4864.132347] LustreError: 171916:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4864.145884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.111882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.245459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.068730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.172507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.225392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.134892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.034466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.948275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.761672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.682255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.570624] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4874.575342] Lustre: Skipped 31 previous similar messages [ 4874.586598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4874.594454] Lustre: Skipped 31 previous similar messages [ 4874.598231] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4874.600894] Lustre: Skipped 31 previous similar messages [ 4874.605170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.612561] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4874.614774] Lustre: Skipped 31 previous similar messages [ 4875.739083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.654580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.805353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.600210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.651951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.676765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.854891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.807869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.118095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.136243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.540561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.515971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.363835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.120396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.976717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.052440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.173678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.529911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.680236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.700181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.917416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.021702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.083708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.932343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.004593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.044788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.170778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.116296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.063234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.084705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.167179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.221410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.253416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.212343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.467667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.434436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.395138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.245393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.104340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.059082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.112328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.378440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.498113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.268877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.414713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.699134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.806991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.870747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.719802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.702643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.598105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.621429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.623245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.614891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.448394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.269359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.167320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.098062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.057897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.139693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.111936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.050581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.040754] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4939.047027] Lustre: Skipped 62 previous similar messages [ 4939.064322] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4939.072133] Lustre: Skipped 62 previous similar messages [ 4939.076055] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4939.080038] Lustre: Skipped 62 previous similar messages [ 4939.083225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.096219] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4939.103280] Lustre: Skipped 62 previous similar messages [ 4939.946103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.704882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.836596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.774625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.679873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.501075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.630700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.629755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.630793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.619068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.610920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.590080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.549745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.689717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.852755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.891805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.901423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.052672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.063623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.978570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.873163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.859498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.691271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.863268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.670666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.475713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.463568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.259795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.092772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.058471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.992764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.111579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.274785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.110410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.165403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.013676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.060404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.000800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.191461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.329879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.269210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.215499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.197430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.277878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.199365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.063600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.862434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.867266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.834614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.679979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.761400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.806729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.785298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.981543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.908756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.904717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.955378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.074406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.185576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.981268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.058807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.836469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.798085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.714349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.591376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.493212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.444136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.329684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.355715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.248244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.175257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.999382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.801353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.620928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.499803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.602725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.641903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.759128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.845228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.759891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.693667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.466480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.390125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.210785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.211440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.925506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.664854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.619767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.484396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.277205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.088609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.031918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.003271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.079448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.090676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.331160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.365536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.478966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.480498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.406053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.457365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.573531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.705335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.819055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.912426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.811492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.889242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.988718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.060875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.989398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.936906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.874825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.983559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.860103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.923582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.722118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.813586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.784320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.701262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.623544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.518904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.340474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.167038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.098587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.035736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.995118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.952669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.913261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.834076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.872187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.778945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.929906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.802681] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5067.808113] Lustre: Skipped 132 previous similar messages [ 5067.831548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5067.842678] Lustre: Skipped 132 previous similar messages [ 5067.846415] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5067.851940] Lustre: Skipped 132 previous similar messages [ 5067.856212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.872719] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5067.880471] Lustre: Skipped 132 previous similar messages [ 5068.803333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.839226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.859181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.852875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.972241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.246223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.221173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.196849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.157668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.050323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.754098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.572277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.448940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.383863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.196746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.183594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.266778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.039817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.769481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.556428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.321302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.134613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.011336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.125152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.097284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.098539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.510785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.822272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.731861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.770337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.830270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.847619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.978669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.050231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.333077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.627256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.626167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.675631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.524448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.463834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.458836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.376218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.266912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.201196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.132360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.051049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.944461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.850169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.772830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.887743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.808941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.737872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.940488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.122069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.140225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.155338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.094846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.988690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.901174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.131693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.110950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.296960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.245869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.513753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.673472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.678800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.847106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.993303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.093314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.207672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.345424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.326890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.339374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.263659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.366168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.366433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.309960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.200040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.294650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.322411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.422817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.345246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.445073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.527234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.658320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.744708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.829703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.979678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.188647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.413799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.951835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.070951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.347429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.314441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.717217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.695576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.799929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.620027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.500799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.584816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.579339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.558831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.445177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.301887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.423275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.393181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.598217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.710575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.636313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.498192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.476770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.676878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.580784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.589797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.563530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.445595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.200050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.135706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.988825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.919657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.969463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.132354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.040987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.018851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.208823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.216634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.185347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.222926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.192212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.256122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.239770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.263039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.252750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.159338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.083181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.063942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.145670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.977433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.082448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.141703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.171150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.251459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.388515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.302620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.329048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.469564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.778856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.082585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.315489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.251460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.489348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.573917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.509193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.569497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.753981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.623300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.728802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.847529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.942337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.903916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.028241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.066319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.061504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.070755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.982195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.940303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.772870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.546364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.360458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.305195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.168866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.995566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.964072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.011086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.205921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.185648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.238228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.141478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.129500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.942574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.915862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.939925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.858550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.929978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.229695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.214730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.211117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.960419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.906914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.879355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.821361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.866432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.058479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.058356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.193409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.361955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.647284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.946959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.983746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.182972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.155171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.325558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.100663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.483189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.508867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.779593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.053810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.345247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.774514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.836576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.796727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.022075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.076035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.035945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.965704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.986385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.973715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.039518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.156485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.220632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.335448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.421317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.491957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.288799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.361128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.381248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.260704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.049079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.180864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.600037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.930102] LustreError: 171916:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5305.950474] LustreError: 171916:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5305.955350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.138248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.103724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.005546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.266985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.355629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.760031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.211772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.445608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.581683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.572074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.669070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.641624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.824580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.416295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.062366] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5324.072185] Lustre: Skipped 245 previous similar messages [ 5324.095917] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5324.106977] Lustre: Skipped 245 previous similar messages [ 5324.112406] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5324.118273] Lustre: Skipped 245 previous similar messages [ 5324.124373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.139436] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5324.147206] Lustre: Skipped 245 previous similar messages [ 5325.444681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.620509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.074418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.246548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.170610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.092643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.003683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.890468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.153779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.294739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.610747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.542785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.715282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.969756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.134231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.346567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.748154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.395112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.866087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.991783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.870399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.932944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.833901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.821996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.811318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.057088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.002610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.034566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.000410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.010346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.927762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.915074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.860669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.676813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.661991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.662867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.746507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.804761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.986241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.999943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.972583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.867803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.853905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.908264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.966665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.088791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.363268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.559689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.759727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.914567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.061357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.078469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.224364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.276812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.205535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.386963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.546698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.450073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.476395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.413787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.324240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.149775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.135939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.018852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.139274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.196515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.093635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.186365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.244115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.075355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.129312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.994281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.955778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.942637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.881490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.960648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.031131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.883563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.355446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.618480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.536985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.490884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.543127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.391245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.496741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.489775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.490858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.449441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.437739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.585420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.482522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.387272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.397407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.334804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.717518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.740621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.652920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.608059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.619487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.601040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.765551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.604441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.649205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.624695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.495187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.581462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.443276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.553309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.450248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.395079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.278636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.300372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.315602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.322860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.323805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.154645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.981518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.893160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.839418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.652753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.592192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.428279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.392915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.338338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.220542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.357793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.449343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.353607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.590539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.669841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.613467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.657631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.821375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.816051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.693332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.701021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.645644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.479041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.422595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.306868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.236916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.164752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.099984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.105140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.080068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.874691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.686606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.617097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.449896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.208887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.151181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.158914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.112920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.093625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.948497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.991922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.975525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.907386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.961492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.908738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.805923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.533480] LustreError: 171916:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5488.538321] LustreError: 171916:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5488.547348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.319973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.563423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.452394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.383575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.305811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.320100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.070366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.940285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.897228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.811780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.655450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.718513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.710110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.644870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.447712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.562126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.578248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.550695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.429295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.388467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.375199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.363685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.281491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.105153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.023532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.148622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.145594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.254758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.159807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.209166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.986733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.045482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.960455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.948428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.992458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.040748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.262775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.260138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.270465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.236035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.304790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.268092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.303349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.316560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.316884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.451833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.466464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.657615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.649529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.701334] LustreError: 171916:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5537.711613] LustreError: 171916:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5537.724356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.785768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.861551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.779873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.848085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.893193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.002540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.997309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.120438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.210410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.260778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.390197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.502749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.449964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.608112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.805797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.815901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.837712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.675784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.944703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.795135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.755599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.753965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.756936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.935380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.060144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.137895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.296213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.309696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.405976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.259965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.241412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.055768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.908556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.940920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.837501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.797411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.784090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.660335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.578801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.519203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.337548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.097145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.294276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.070197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.063883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.135236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.192246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.214984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.317800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.337539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.302909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.240187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.171218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.069086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.949573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.814442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.575177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.358603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.244648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.254608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.198505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.085604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.033437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.005975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.022742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.940613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.954933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.896581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.779246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.708360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.609844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.590403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.708695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.610440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.462553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.458907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.430718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.451245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.380993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.336450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.177129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.078253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.941956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.904335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.787953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.850490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.821564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.752517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.586524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.676317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.521590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.662123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.570588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.604691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.696153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.664366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.716419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.711673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.816759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.682983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.581746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.277435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.101645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.227907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.285357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.057293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.074760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.099806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.856191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.763895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.666222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.736524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.705760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.734386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.751271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.624904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.455115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.337376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.153399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.024722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.857981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.829950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.674491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.555356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.410976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.448853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.404498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.390436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.509915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.834629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.019839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.823567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.799833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.810797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.809752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.799770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.915305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.811713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.831246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.126227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.262823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.398898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.511645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.519556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.533531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.280933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.637043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.763233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.652336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.706238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.693796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.833477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.097781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.998759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.979764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.143363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.926159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.961381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.781840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.814536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.823937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.004155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.856163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.972420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.838710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.800625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.635807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.449222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.225948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.069544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.773257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.644096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.665302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.470147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.468442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.307106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.161353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.236702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.161320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.218270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.239918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.272066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.183587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.223975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.151939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.172796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.130559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.191935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.232409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.132623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.070951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.098762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.094967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.224887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.255601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.318804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.588586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.673941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.629776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.934886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.893917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.984716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.135495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.248359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.487072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.500566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.300902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.091274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.850385] LustreError: 171916:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5742.863808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.877498] LustreError: 171916:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5743.819286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.865689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.635564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.807564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.930608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.951512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.097273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.216774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.156770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.421928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.725263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.733602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.118834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.462387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.505715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.612766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.746342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.154251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.208490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.197153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.194864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.247575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.158087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.015104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.833724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.092529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.145051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.068467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.133602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.048705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.010305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.029905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.949995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.819836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.829760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.709263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.842591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.879473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.762062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.621976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.524491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.559773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.573769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.546451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.508695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.398833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.300667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.241050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.154987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.195640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.165073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.324689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.347845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.246062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.169871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.116098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.026644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.031664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.079382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.043422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.045711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.165229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.251061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.338483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.432407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.609514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.624755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.408080] LustreError: 173719:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6218.429216] LustreError: 171916:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6941.155822] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6941.175235] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6941.196411] Lustre: Skipped 488 previous similar messages [ 6941.205284] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6941.668557] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6942.689783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6942.700179] Lustre: Skipped 2 previous similar messages [ 6946.084528] Lustre: server umount lustre-OST0000 complete [ 6952.934274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6952.945993] Lustre: Skipped 1 previous similar message [ 6958.052039] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6958.056889] Lustre: Skipped 1 previous similar message [ 6963.168926] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6963.173842] Lustre: Skipped 1 previous similar message [ 6964.197073] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6964.599662] Lustre: server umount lustre-MDT0000 complete [ 6967.266137] LustreError: 220310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6967.280684] LustreError: 220310:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6968.002664] LustreError: 171876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799153 with bad export cookie 10304828124037588414 [ 6968.009777] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6968.012068] LustreError: 171876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6968.294499] Lustre: server umount lustre-MDT0001 complete [ 6978.446409] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 6981.910947] Key type lgssc unregistered [ 6982.231393] LNet: 227175:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6982.236352] LNetError: 227175:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6982.258041] LNet: Removed LNI 192.168.206.116@tcp [ 6983.210859] Key type .llcrypt unregistered [ 6983.213701] Key type ._llcrypt unregistered [ 7006.518761] Key type ._llcrypt registered [ 7006.521666] Key type .llcrypt registered [ 7006.640075] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7008.022581] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7008.050181] alg: No test for adler32 (adler32-zlib) [ 7009.130191] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 7009.349891] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 7011.007212] Key type lgssc registered [ 7012.191418] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7019.859085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7019.891535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7021.402471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7025.806507] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7031.780620] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7031.789305] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7035.357450] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7035.683103] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7040.215841] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7047.305593] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7052.926437] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7059.962590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7060.303437] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7066.520592] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7066.791608] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7075.245823] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7080.049349] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7081.887169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7081.897287] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7081.903624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7081.913030] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7081.917587] LustreError: 228606:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7087.078957] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7087.079413] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7087.097235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7087.106831] Lustre: Skipped 1 previous similar message [ 7092.202963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7092.217578] Lustre: Skipped 3 previous similar messages [ 7097.316359] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7097.320978] Lustre: Skipped 2 previous similar messages [ 7098.335290] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7098.633142] Lustre: server umount lustre-OST0000 complete [ 7101.999862] LustreError: 228606:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7133.889291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7134.280446] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7141.216548] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7148.861247] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7152.748376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7152.776591] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7152.784113] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7152.805898] Lustre: Skipped 1 previous similar message [ 7154.497539] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 7160.654315] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff916e0758c000.ost_server_uuid 50 [ 7161.885401] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff916e0758c000.ost_server_uuid in FULL state after 0 sec [ 7168.149038] Lustre: server umount lustre-OST0000 complete [ 7173.600496] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7173.605459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7173.613321] Lustre: Skipped 1 previous similar message [ 7173.623285] Lustre: Skipped 2 previous similar messages [ 7178.721849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7178.728835] Lustre: Skipped 1 previous similar message [ 7181.851160] Lustre: server umount lustre-MDT0000 complete [ 7183.843225] LustreError: 228581:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7183.865978] LustreError: 228581:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7184.873265] Lustre: 227896:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787799354/real 1787799354] req@000000008939fdf8 x1874643328750976/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787799370 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7184.879745] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7184.903883] Lustre: 227896:0:(client.c:2503:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 7184.923229] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7184.956749] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7185.644985] LustreError: 228567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799371 with bad export cookie 16788991363539775330 [ 7185.658679] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7185.662604] LustreError: 228567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7185.882396] Lustre: server umount lustre-MDT0001 complete [ 7194.631374] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 7197.140603] Key type lgssc unregistered [ 7197.451798] LNet: 233793:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7197.472226] LNetError: 233793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7197.495954] LNet: Removed LNI 192.168.206.116@tcp [ 7198.210734] Key type .llcrypt unregistered [ 7198.215406] Key type ._llcrypt unregistered [ 7225.816559] Key type ._llcrypt registered [ 7225.818720] Key type .llcrypt registered [ 7225.894682] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7226.878748] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7226.914191] alg: No test for adler32 (adler32-zlib) [ 7227.930456] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 7228.118902] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 7229.767228] Key type lgssc registered [ 7230.707278] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7238.327820] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7238.351682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7239.731605] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7243.680781] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7249.898391] LustreError: 235518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7249.920462] LustreError: 235518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7253.232985] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7253.541511] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7258.218628] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7260.822520] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7266.785693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7267.124477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7272.781095] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7280.447095] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7280.716846] Lustre: lustre-OST0001: new disk, initializing [ 7280.721078] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7280.738995] Lustre: 237827:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7280.849533] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7285.000712] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7286.389866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7286.413899] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7286.549090] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7286.767613] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7297.655898] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7307.700261] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7320.035273] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7320.050277] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7320.066273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7321.176684] Lustre: server umount lustre-MDT0000 complete [ 7322.613701] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7322.618815] LustreError: 235518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7322.656759] LustreError: 235518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7324.633554] LustreError: 237825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799510 with bad export cookie 14463938967316598786 [ 7324.635848] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7324.644063] LustreError: 237825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7324.919859] Lustre: server umount lustre-MDT0001 complete [ 7337.411069] Lustre: server umount lustre-OST0000 complete [ 7351.845141] Lustre: server umount lustre-OST0001 complete [ 7372.934214] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7384.179325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7384.809173] LustreError: 240933:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7384.940659] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7389.523309] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7390.182901] LustreError: 240934:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7394.279130] LustreError: 240933:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7399.316760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7399.793595] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7404.819601] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7407.909431] Lustre: 242074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7414.870134] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7422.097993] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7424.556707] LustreError: 242428:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7424.590290] LustreError: 242428:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7430.658402] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7431.510935] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7431.788785] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7431.795695] Lustre: Skipped 1 previous similar message [ 7437.301994] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7437.516413] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7443.934389] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7447.187379] Lustre: 243947:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7455.787799] Lustre: server umount mylustre-MDT0000 complete [ 7456.224020] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7456.230282] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7456.242475] Lustre: Skipped 2 previous similar messages [ 7456.257841] LustreError: 240933:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7456.276157] LustreError: 240933:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7459.142967] LustreError: 240908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799644 with bad export cookie 14463938967316601593 [ 7459.153419] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7459.157019] LustreError: 240908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7461.588223] Lustre: server umount mylustre-MDT0001 complete [ 7474.506064] Lustre: server umount mylustre-OST0000 complete [ 7487.597214] Lustre: server umount mylustre-OST0001 complete [ 7513.916916] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7525.040039] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7525.376764] LustreError: 246266:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7525.394721] LustreError: 246266:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 7525.453522] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7529.334515] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7538.358667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7543.380622] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7546.160923] Lustre: 247406:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7552.676280] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7559.207476] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7567.794514] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7571.153674] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7571.160556] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7573.803548] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7581.406711] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7585.142683] Lustre: 249283:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7594.983182] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7594.999962] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7595.010503] Lustre: Skipped 3 previous similar messages [ 7595.016727] LustreError: 246266:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7595.044804] LustreError: 246266:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7596.441257] Lustre: server umount tfs-MDT0000 complete [ 7600.446299] LustreError: 246243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799786 with bad export cookie 14463938967316603483 [ 7600.461303] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7600.463727] LustreError: 246243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7600.978124] Lustre: server umount tfs-MDT0001 complete [ 7615.345965] Lustre: server umount tfs-OST0000 complete [ 7617.252744] Lustre: 234779:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787799787/real 1787799787] req@ffff922d23b18000 x1874643557481984/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787799803 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7617.277743] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7617.293152] Lustre: Skipped 3 previous similar messages [ 7619.771340] Lustre: server umount tfs-OST0001 complete [ 7640.710885] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7650.727285] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7651.239764] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7651.247378] Lustre: Skipped 3 previous similar messages [ 7655.999803] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7664.904029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7670.280620] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7673.391339] Lustre: 252674:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7680.093243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7686.354140] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7695.354896] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7696.738690] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7696.744741] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7702.081485] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7709.927939] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7714.117612] Lustre: 254546:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7722.495196] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:05:06 (1787799906) [ 7741.920580] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7741.926110] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7741.948696] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7742.956416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7748.079523] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7748.089590] Lustre: Skipped 6 previous similar messages [ 7753.183185] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7753.186291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7753.205995] Lustre: Skipped 3 previous similar messages [ 7753.368477] Lustre: server umount lustre-MDT0000 complete [ 7756.937992] LustreError: 254549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799942 with bad export cookie 14463938967316605149 [ 7756.945769] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7756.950027] LustreError: 254549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7758.306185] LustreError: 253049:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7758.306386] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7758.327720] LustreError: 253049:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 7758.348838] Lustre: Skipped 4 previous similar messages [ 7761.316605] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7761.324116] Lustre: Skipped 2 previous similar messages [ 7763.480810] Lustre: server umount lustre-MDT0001 complete [ 7773.557397] Lustre: server umount lustre-OST0000 complete [ 7783.482794] Lustre: server umount lustre-OST0001 complete [ 7789.724943] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 7799.807582] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7835.487977] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7835.702784] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7835.724094] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7835.780411] Lustre: lustre-MDT0000: new disk, initializing [ 7835.893888] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7835.901728] Lustre: Skipped 3 previous similar messages [ 7835.912259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7839.831830] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7849.684972] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7849.798611] Lustre: 257731:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7849.839481] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7849.843462] Lustre: Skipped 1 previous similar message [ 7849.916699] Lustre: lustre-MDT0001: new disk, initializing [ 7850.032387] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7850.043857] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7854.579390] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7864.391606] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7864.627610] Lustre: lustre-OST0000: new disk, initializing [ 7864.630555] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7864.635068] Lustre: 259313:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7865.801839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7865.812542] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7865.941622] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7869.949418] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7878.914056] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7879.040823] Lustre: lustre-OST0001: new disk, initializing [ 7879.042750] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7879.046947] Lustre: 260185:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7880.202400] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7880.211271] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7880.320476] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7885.748124] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7898.184755] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 23:08:02 (1787800082) [ 7915.941682] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 23:08:20 (1787800100) [ 7932.390445] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7932.393870] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7932.399483] LustreError: Skipped 1 previous similar message [ 7932.402105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7932.402194] Lustre: Skipped 1 previous similar message [ 7932.426854] Lustre: Skipped 1 previous similar message [ 7945.183193] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7945.305801] Lustre: server umount lustre-OST0000 complete [ 7957.277238] Lustre: server umount lustre-MDT0000 complete [ 7960.617518] LustreError: 257724:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800146 with bad export cookie 14463938967316606703 [ 7960.618894] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7960.634822] LustreError: 257724:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7965.156302] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7965.159192] Lustre: Skipped 13 previous similar messages [ 7967.046331] Lustre: server umount lustre-MDT0001 complete [ 7977.796994] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 8050.241361] Lustre: server umount lustre-OST0001 complete [ 8056.421827] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 8071.550471] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8108.357918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8108.639699] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8108.668138] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8108.731955] Lustre: lustre-MDT0000: new disk, initializing [ 8108.792136] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8108.797533] Lustre: Skipped 3 previous similar messages [ 8108.826721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8113.209347] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8123.556597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8123.685748] Lustre: 265038:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8123.955619] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8128.794166] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8138.021665] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8143.340481] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8149.811731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8150.034279] Lustre: lustre-OST0000: new disk, initializing [ 8150.037371] Lustre: Skipped 1 previous similar message [ 8150.041710] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8150.046495] Lustre: Skipped 2 previous similar messages [ 8150.052248] Lustre: 267004:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8152.026908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8152.036784] Lustre: Skipped 1 previous similar message [ 8152.041475] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8152.176096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8156.034952] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8164.782197] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8172.513567] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8172.514625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8172.527727] LustreError: Skipped 1 previous similar message [ 8172.530648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8172.545742] Lustre: Skipped 5 previous similar messages [ 8172.566995] Lustre: Skipped 1 previous similar message [ 8177.113926] Lustre: server umount lustre-OST0000 complete [ 8186.394085] Lustre: server umount lustre-MDT0000 complete [ 8187.874164] LustreError: 265046:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8187.888711] LustreError: 265046:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8189.278886] LustreError: 265030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800375 with bad export cookie 14463938967316608166 [ 8189.283524] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8189.288241] LustreError: 265030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8189.560497] Lustre: server umount lustre-MDT0001 complete [ 8199.135416] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 8201.802477] Key type lgssc unregistered [ 8202.149768] LNet: 268763:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8202.164237] LNetError: 268763:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8202.186456] LNet: Removed LNI 192.168.206.116@tcp [ 8202.943180] Key type .llcrypt unregistered [ 8202.947544] Key type ._llcrypt unregistered [ 8226.181295] Key type ._llcrypt registered [ 8226.184315] Key type .llcrypt registered [ 8226.264670] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8226.985532] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8227.119186] alg: No test for adler32 (adler32-zlib) [ 8228.075532] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 8228.270846] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 8229.943161] Key type lgssc registered [ 8230.805110] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8237.397862] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8237.420982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8238.776226] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8242.714359] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8248.293522] LustreError: 270224:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8248.309023] LustreError: 270224:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8249.812667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8250.161869] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8254.606707] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8261.693574] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8267.164983] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8273.069494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8273.436173] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8278.328876] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8283.704896] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8284.376782] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8285.213654] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8288.743629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8288.750747] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8288.763899] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8288.783765] Lustre: Skipped 1 previous similar message [ 8293.116979] Lustre: server umount lustre-OST0000 complete [ 8298.978029] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8298.982025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8298.987982] Lustre: Skipped 2 previous similar messages [ 8302.824618] Lustre: server umount lustre-MDT0000 complete [ 8304.098270] LustreError: 271247:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8304.120072] LustreError: 271247:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8306.879619] LustreError: 270205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800492 with bad export cookie 15920712733909098805 [ 8306.890053] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8306.901824] LustreError: 270205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8307.622201] Lustre: server umount lustre-MDT0001 complete [ 8319.246792] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 8322.051938] Key type lgssc unregistered [ 8322.304540] LNet: 273805:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8322.313903] LNetError: 273805:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8322.338237] LNet: Removed LNI 192.168.206.116@tcp [ 8323.193238] Key type .llcrypt unregistered [ 8323.195060] Key type ._llcrypt unregistered [ 8339.701426] Key type ._llcrypt registered [ 8339.703426] Key type .llcrypt registered [ 8339.820768] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8340.619523] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8340.798277] alg: No test for adler32 (adler32-zlib) [ 8341.910029] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 8342.179327] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 8343.871195] Key type lgssc registered [ 8345.141607] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8354.522482] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8354.557529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8370.143427] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8375.263741] LustreError: 274995:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.116@tcp: failed processing log, type 4: rc = -110 [ 8401.887194] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8407.368271] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8414.032732] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8416.402324] Lustre: server umount lustre-OST0000 complete [ 8431.281280] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 8433.988394] Key type lgssc unregistered [ 8434.287699] LNet: 276625:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8434.295763] LNetError: 276625:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8434.310633] LNet: Removed LNI 192.168.206.116@tcp [ 8435.033330] Key type .llcrypt unregistered [ 8435.038210] Key type ._llcrypt unregistered [ 8473.204230] Key type ._llcrypt registered [ 8473.208538] Key type .llcrypt registered [ 8473.357882] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8474.540436] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8474.619933] alg: No test for adler32 (adler32-zlib) [ 8475.756126] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 8475.998027] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 8477.791216] Key type lgssc registered [ 8479.125250] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8490.956476] loop0: detected capacity change from 0 to 204800000 [ 8491.131302] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8491.146210] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8491.470277] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8492.649149] loop0: detected capacity change from 0 to 204800000 [ 8492.859428] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8492.875234] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8493.312590] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8494.366481] loop0: detected capacity change from 0 to 204800000 [ 8494.532772] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8494.557960] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8494.723705] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8496.077033] loop0: detected capacity change from 0 to 204800000 [ 8496.301645] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8496.310732] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8496.424957] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8497.461897] loop0: detected capacity change from 0 to 204800000 [ 8497.494597] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8501.226397] loop0: detected capacity change from 0 to 204800000 [ 8501.267210] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8505.242457] loop0: detected capacity change from 0 to 204800000 [ 8505.272270] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8509.303556] loop0: detected capacity change from 0 to 204800000 [ 8509.327361] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8513.308163] loop0: detected capacity change from 0 to 204800000 [ 8513.385900] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8513.425242] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8513.491658] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8513.500638] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8513.509194] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8513.518050] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8513.527149] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8513.535115] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8513.543887] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8513.553324] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8513.567796] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8513.591721] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8513.614391] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8513.637237] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8513.654518] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8513.671315] Lustre: 280037:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8519.306900] Lustre: server umount MGS complete [ 8520.149773] loop0: detected capacity change from 0 to 204800000 [ 8520.240358] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8520.679366] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8520.762540] Lustre: lustre-MDT0000: Aborting client recovery [ 8521.839970] loop1: detected capacity change from 0 to 204800000 [ 8521.986681] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8522.118113] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8522.144533] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8522.158762] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8522.170747] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8522.187865] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8522.214783] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8522.227588] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8522.248302] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8522.392759] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8522.455645] Lustre: lustre-MDT0001: Aborting client recovery [ 8523.301310] loop2: detected capacity change from 0 to 204800000 [ 8523.446405] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8523.521262] Lustre: 280547:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8523.548423] Lustre: 280547:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8523.584417] Lustre: 280547:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8523.595329] Lustre: 280547:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8523.613075] Lustre: 280547:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8523.627612] Lustre: 280547:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8523.639870] Lustre: 280547:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8523.769466] Lustre: Found index 0 for lustre-OST0000, updating log [ 8523.800724] Lustre: 280565:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8523.890468] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8523.911993] Lustre: lustre-OST0000: Aborting recovery [ 8524.852234] loop3: detected capacity change from 0 to 204800000 [ 8524.954924] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8525.089817] Lustre: 280630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8525.110163] Lustre: 280630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8525.121529] Lustre: 280630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8525.135863] Lustre: 280630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8525.144440] Lustre: 280630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8525.154197] Lustre: 280630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8525.173235] Lustre: 280630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8525.196322] Lustre: Found index 1 for lustre-OST0001, updating log [ 8525.228248] Lustre: 280635:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8525.362045] Lustre: lustre-OST0001: Aborting recovery [ 8530.460709] LustreError: 280410:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8531.434752] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8531.473403] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8531.490460] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8531.533252] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8532.450640] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8532.454724] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8533.204608] LustreError: 280377:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1005us: rc = -2 [ 8533.716859] LustreError: 280374:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3366us: rc = -2 [ 8533.747993] LustreError: 280374:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 16 previous similar messages [ 8547.808317] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8547.812556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8547.820389] Lustre: Skipped 2 previous similar messages [ 8547.825226] Lustre: Skipped 1 previous similar message [ 8552.899388] Lustre: server umount lustre-MDT0000 complete [ 8552.931115] LustreError: 280379:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8552.946559] LustreError: 280379:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8553.642480] LustreError: 280362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800739 with bad export cookie 10242338301181298261 [ 8553.646655] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8553.655894] LustreError: 280362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8558.047753] LustreError: 280374:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8558.054448] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8558.061362] LustreError: 280374:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8558.062571] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8558.072404] Lustre: Skipped 2 previous similar messages [ 8558.083340] Lustre: Skipped 2 previous similar messages [ 8560.023406] Lustre: server umount lustre-MDT0001 complete [ 8566.941697] Lustre: server umount lustre-OST0000 complete [ 8582.111918] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8582.288291] Lustre: server umount lustre-OST0001 complete [ 8589.398647] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:19:34 (1787800774) [ 8615.402244] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 8624.256496] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8655.843049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8656.114777] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8656.152285] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8656.221623] Lustre: lustre-MDT0000: new disk, initializing [ 8656.278864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8656.286342] Lustre: Skipped 1 previous similar message [ 8656.297254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8660.254540] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8671.013135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8671.187172] Lustre: 284706:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8671.281762] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8671.286185] Lustre: Skipped 1 previous similar message [ 8671.387753] Lustre: lustre-MDT0001: new disk, initializing [ 8671.491311] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8671.520864] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8671.530278] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8676.150863] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8685.698571] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8691.390527] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8697.803258] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8698.055788] Lustre: lustre-OST0000: new disk, initializing [ 8698.079439] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8698.086635] Lustre: 286676:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8698.156476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8703.588095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8703.605488] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8703.741919] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8704.211036] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8714.969585] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8720.560594] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8733.987716] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8738.189441] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8744.875304] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8791.010154] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8791.013427] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8791.015313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8791.015320] Lustre: Skipped 1 previous similar message [ 8791.024197] LustreError: Skipped 1 previous similar message [ 8791.037897] Lustre: Skipped 1 previous similar message [ 8796.133869] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8796.141628] Lustre: Skipped 2 previous similar messages [ 8801.247129] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8801.254549] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8801.265354] Lustre: Skipped 1 previous similar message [ 8801.444406] Lustre: server umount lustre-OST0000 complete [ 8806.377928] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8806.399358] Lustre: Skipped 1 previous similar message [ 8811.372322] Lustre: server umount lustre-MDT0000 complete [ 8811.499538] LustreError: 284718:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8811.515200] LustreError: 284718:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8814.834885] LustreError: 286675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801000 with bad export cookie 10242338301181300977 [ 8814.844351] LustreError: 286675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8814.846494] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8815.298856] Lustre: server umount lustre-MDT0001 complete [ 8823.380163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8827.211732] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8831.588873] Lustre: server umount MGS complete [ 8843.462769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8843.965991] LustreError: 290552:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8843.984890] LustreError: 290565:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8844.027934] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8849.079977] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8849.384361] LustreError: 290553:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8849.403741] LustreError: 290553:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8857.349704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8863.403980] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8873.106655] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8882.349072] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8891.087656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8891.532280] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8891.541810] Lustre: Skipped 1 previous similar message [ 8899.513626] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8910.590954] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8944.099841] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8944.104839] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8944.150216] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8944.157405] Lustre: Skipped 2 previous similar messages [ 8945.126664] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8950.469045] Lustre: server umount lustre-OST0000 complete [ 8954.853598] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8954.862959] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8954.876381] Lustre: Skipped 1 previous similar message [ 8960.405896] Lustre: server umount lustre-MDT0000 complete [ 8963.556840] LustreError: 290553:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8963.593537] LustreError: 290553:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8964.194668] LustreError: 290533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801150 with bad export cookie 10242338301181302902 [ 8964.210106] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8964.215783] LustreError: 290533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8964.662921] Lustre: server umount lustre-MDT0001 complete [ 8975.999397] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 8979.601968] Key type lgssc unregistered [ 8979.961958] LNet: 294499:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8979.970714] LNetError: 294499:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8981.037856] LNet: Removed LNI 192.168.206.116@tcp [ 8982.086240] Key type .llcrypt unregistered [ 8982.090446] Key type ._llcrypt unregistered [ 9019.499399] Key type ._llcrypt registered [ 9019.503854] Key type .llcrypt registered [ 9019.619122] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 9037.491434] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 9038.723347] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9038.796996] alg: No test for adler32 (adler32-zlib) [ 9040.077090] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 9040.434128] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 9042.151178] Key type lgssc registered [ 9043.484963] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9082.612564] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9082.644500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9084.048981] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9084.093555] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9084.236822] Lustre: lustre-MDT0000: new disk, initializing [ 9084.355234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9084.389486] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9088.720535] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9099.074267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9099.223161] Lustre: 298453:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9099.291885] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9099.295420] Lustre: Skipped 1 previous similar message [ 9099.372967] Lustre: lustre-MDT0001: new disk, initializing [ 9099.471340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9099.518536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9099.525863] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9103.938368] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9113.271453] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9119.151188] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9126.120414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9126.435417] Lustre: lustre-OST0000: new disk, initializing [ 9126.439863] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9126.448670] Lustre: 300422:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9126.570219] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9132.747746] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9133.634551] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9133.650137] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9133.809982] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9141.859577] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9148.972514] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9161.501273] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9167.306340] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9210.849264] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9210.852194] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9210.859900] LustreError: Skipped 1 previous similar message [ 9210.880441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9210.886425] Lustre: Skipped 1 previous similar message [ 9215.974348] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9215.980903] Lustre: Skipped 1 previous similar message [ 9216.795595] Lustre: server umount lustre-OST0000 complete [ 9221.089327] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9221.091779] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9221.111180] Lustre: Skipped 2 previous similar messages [ 9221.123313] Lustre: Skipped 1 previous similar message [ 9226.209474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9226.217809] Lustre: Skipped 1 previous similar message [ 9227.388743] Lustre: server umount lustre-MDT0000 complete [ 9231.403990] LustreError: 298446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801417 with bad export cookie 9751415927349294002 [ 9231.408569] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9231.428053] LustreError: 298446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9231.909601] Lustre: server umount lustre-MDT0001 complete [ 9240.715545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9245.819660] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9250.923537] Lustre: server umount MGS complete [ 9262.433430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9262.726716] LustreError: 304336:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9262.737636] LustreError: 304336:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9262.744420] LustreError: 304349:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9262.792308] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9267.693799] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9268.192485] LustreError: 304337:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9272.290026] LustreError: 304336:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9276.452325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9276.898893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9281.444734] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9288.732539] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9296.203494] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9305.344873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9305.637499] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9313.476672] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9322.814971] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9348.578351] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9348.591416] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9348.610750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9352.678844] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9352.682573] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9354.231309] Lustre: server umount lustre-OST0000 complete [ 9358.817086] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9358.824789] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9358.847675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9358.866318] Lustre: Skipped 2 previous similar messages [ 9364.288926] Lustre: server umount lustre-MDT0000 complete [ 9368.011512] LustreError: 307063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801553 with bad export cookie 9751415927349295843 [ 9368.022933] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9368.024410] LustreError: 307063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9368.441382] Lustre: server umount lustre-MDT0001 complete [ 9380.738819] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 9383.615714] Key type lgssc unregistered [ 9384.113764] LNet: 308279:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9384.118671] LNetError: 308279:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9384.141484] LNet: Removed LNI 192.168.206.116@tcp [ 9385.265536] Key type .llcrypt unregistered [ 9385.266941] Key type ._llcrypt unregistered [ 9406.489438] Key type ._llcrypt registered [ 9406.491486] Key type .llcrypt registered [ 9406.613208] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 9422.318438] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 9423.602921] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9423.692790] alg: No test for adler32 (adler32-zlib) [ 9424.745927] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [ 9424.993753] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 9426.663472] Key type lgssc registered [ 9427.945952] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9453.172259] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 9467.262430] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9467.288980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9468.502085] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9468.548116] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9468.621854] Lustre: lustre-MDT0000: new disk, initializing [ 9468.687527] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9468.703951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9473.308639] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9483.906690] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9484.179540] Lustre: lustre-OST0000: new disk, initializing [ 9484.182460] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9484.187428] Lustre: Skipped 1 previous similar message [ 9484.192258] Lustre: 312328:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9484.253089] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9485.083834] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9485.092584] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9485.118794] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9490.119823] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10412.118680] Lustre: server umount lustre-MDT0000 complete [10416.224869] LustreError: 311514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802602 with bad export cookie 229153418141431431 [10416.231158] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10427.061331] Lustre: server umount lustre-OST0000 complete [10454.468806] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [10466.640953] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [10511.813508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10512.214274] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10512.240191] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10512.296598] Lustre: lustre-MDT0000: new disk, initializing [10512.355170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10512.393237] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10517.697206] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10530.141134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10530.369124] Lustre: 316678:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10530.488639] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10530.496451] Lustre: Skipped 1 previous similar message [10530.681251] Lustre: lustre-MDT0001: new disk, initializing [10530.866274] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10530.953284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10530.976271] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10536.447940] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10546.588600] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10552.415869] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10559.217915] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10559.505327] Lustre: lustre-OST0000: new disk, initializing [10559.509942] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10559.518837] Lustre: 318646:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10559.642381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10566.368298] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10569.317705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10569.343292] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10569.518913] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10575.226663] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10579.637662] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10579.817100] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10584.345146] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10584.519937] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10590.179068] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10590.199791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10590.227718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10590.234168] Lustre: Skipped 1 previous similar message [10592.205787] Lustre: server umount lustre-OST0000 complete [10597.344607] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10597.357109] LustreError: Skipped 1 previous similar message [10597.360968] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10597.374221] Lustre: Skipped 1 previous similar message [10597.381911] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10600.421021] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10600.426710] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10601.686993] Lustre: server umount lustre-MDT0000 complete [10605.482322] LustreError: 316671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802791 with bad export cookie 229153418144372250 [10605.486150] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10605.488108] LustreError: 316671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10605.857723] Lustre: server umount lustre-MDT0001 complete [10609.386388] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10611.691193] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 23:53:15 (1787802795) [10619.691354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10620.488897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10624.579447] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10625.510964] LustreError: 320960:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10625.521676] LustreError: 320960:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10630.624487] LustreError: 320959:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10635.745981] LustreError: 320960:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10636.142130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10636.438943] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10642.170164] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10654.311400] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10660.924796] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10668.663528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10668.944818] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10676.136674] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10686.506748] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10696.739637] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10696.926635] Lustre: lustre-OST0001: new disk, initializing [10696.938606] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10696.952291] Lustre: 323797:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10697.100223] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10703.413628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10703.417484] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10703.469628] LustreError: 324101:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10704.001984] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10704.501138] LustreError: 324101:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10704.528309] LustreError: 324101:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10705.575698] LustreError: 324110:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10705.591990] LustreError: 324110:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10707.618460] LustreError: 324101:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10707.635752] LustreError: 324101:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10711.784066] LustreError: 324110:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10711.802489] LustreError: 324110:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10714.209831] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10714.491561] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10719.981151] LustreError: 324101:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10719.996686] LustreError: 324101:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10720.322914] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff916e41afc800.ost_server_uuid 50 [10722.077920] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff916e41afc800.ost_server_uuid in FULL state after 0 sec [10728.626456] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff916e41afc800.ost_server_uuid 50 [10730.068273] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff916e41afc800.ost_server_uuid in FULL state after 0 sec [10731.387749] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10744.806606] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10744.809698] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10744.816820] Lustre: Skipped 1 previous similar message [10744.845843] Lustre: Skipped 2 previous similar messages [10749.922681] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10749.937034] Lustre: Skipped 2 previous similar messages [10750.756253] LustreError: 322783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10750.942750] Lustre: server umount lustre-OST0001 complete [10755.043712] LustreError: 322780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10755.069736] LustreError: 322780:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10758.114409] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10758.123401] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10758.151190] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10763.608127] Lustre: server umount lustre-OST0000 complete [10769.377845] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10769.387771] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10769.400110] Lustre: Skipped 1 previous similar message [10773.463084] Lustre: server umount lustre-MDT0000 complete [10775.524634] LustreError: 320956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10775.557709] LustreError: 320956:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10777.254150] LustreError: 320939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802963 with bad export cookie 229153418144373125 [10777.266555] LustreError: 320939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10777.274854] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10777.643419] Lustre: server umount lustre-MDT0001 complete [10790.456794] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [10793.214736] Key type lgssc unregistered [10793.537708] LNet: 326486:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10793.541820] LNetError: 326486:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10793.559328] LNet: Removed LNI 192.168.206.116@tcp [10794.421952] Key type .llcrypt unregistered [10794.430562] Key type ._llcrypt unregistered [10819.795376] Key type ._llcrypt registered [10819.797810] Key type .llcrypt registered [10819.903611] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [10821.512490] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10821.520319] alg: No test for adler32 (adler32-zlib) [10822.435347] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [10822.563354] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [10824.207208] Key type lgssc registered [10825.445890] Lustre: Echo OBD driver; http://www.lustre.org/ [10833.329264] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10833.354821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10834.932288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10839.612968] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10845.159467] LustreError: 327905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10845.185704] LustreError: 327905:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10846.938454] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10850.276585] LustreError: 327904:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10853.578403] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10854.029724] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10858.079863] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10867.983758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10868.287725] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10874.253852] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10878.525760] LustreError: 329581:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10880.621036] LustreError: 329583:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10880.630694] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10880.640283] LustreError: 329583:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10881.554241] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10886.138710] LustreError: 329583:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10886.179696] LustreError: 329583:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10888.620413] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10888.837166] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10894.362920] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10895.466636] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10904.146372] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10912.835264] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10913.175819] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11027.436953] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11027.441379] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11027.442735] LustreError: Skipped 1 previous similar message [11027.451730] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11027.453342] Lustre: Skipped 1 previous similar message [11027.477785] Lustre: Skipped 1 previous similar message [11031.514128] Lustre: server umount lustre-OST0001 complete [11032.549118] LustreError: 329581:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11032.572359] LustreError: 329581:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11037.667715] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11037.687231] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11042.272480] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11042.283129] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11042.299169] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11043.458455] Lustre: server umount lustre-OST0000 complete [11047.907137] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11047.937637] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11047.961067] Lustre: Skipped 2 previous similar messages [11053.024564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11053.033415] Lustre: Skipped 1 previous similar message [11053.532437] Lustre: server umount lustre-MDT0000 complete [11058.025203] LustreError: 331566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803243 with bad export cookie 6460087133069739408 [11058.028976] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11058.039924] LustreError: 331566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11058.616200] Lustre: server umount lustre-MDT0001 complete [11068.434617] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [11070.586603] Key type lgssc unregistered [11070.820458] LNet: 332846:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11070.830383] LNetError: 332846:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11070.854877] LNet: Removed LNI 192.168.206.116@tcp [11071.554234] Key type .llcrypt unregistered [11071.559991] Key type ._llcrypt unregistered [11124.439598] Key type ._llcrypt registered [11124.441392] Key type .llcrypt registered [11124.542774] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [11125.954476] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11126.005180] alg: No test for adler32 (adler32-zlib) [11127.176486] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [11127.424254] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [11129.095811] Key type lgssc registered [11130.639582] Lustre: Echo OBD driver; http://www.lustre.org/ [11144.675555] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11144.743902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11146.966044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11153.340813] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11156.454992] LustreError: 335152:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11156.464245] LustreError: 335152:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11161.571158] LustreError: 335151:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11163.532182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11164.103205] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11169.081359] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11172.442078] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11178.978234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11179.408626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11186.727163] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11188.644279] LustreError: 336639:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11191.737160] LustreError: 336641:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11191.788779] LustreError: 336641:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11191.789971] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11196.834176] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11196.904376] LustreError: 336641:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11196.930020] LustreError: 336641:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11197.225882] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11202.554380] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11203.453394] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11212.804669] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11223.090867] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11235.809320] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11235.823353] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11235.847919] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11238.374198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11238.376098] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11238.403258] Lustre: Skipped 1 previous similar message [11242.066270] Lustre: server umount lustre-MDT0000 complete [11243.499718] LustreError: 335151:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11243.513545] LustreError: 335151:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11246.266478] LustreError: 336662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803432 with bad export cookie 4612843332795409251 [11246.274409] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11246.280245] LustreError: 336662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11246.608202] Lustre: server umount lustre-MDT0001 complete [11261.019392] Lustre: server umount lustre-OST0000 complete [11274.918151] Lustre: server umount lustre-OST0001 complete [11291.316333] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [11302.132989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11302.740832] LustreError: 340422:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11302.833524] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11307.743414] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11318.333741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11318.763868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11326.472190] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11330.385864] Lustre: 341564:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11336.580304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11336.993295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11342.734191] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11344.182315] LustreError: 341919:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11344.198207] LustreError: 341919:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11347.254199] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11350.428894] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11356.177277] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11356.882662] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11366.186157] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11371.749357] Lustre: 343434:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11380.200500] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11380.219525] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11380.240600] Lustre: Skipped 1 previous similar message [11380.264461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11380.281395] Lustre: Skipped 3 previous similar messages [11382.339540] Lustre: server umount lustre-MDT0000 complete [11387.080117] LustreError: 340403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803572 with bad export cookie 4612843332795410952 [11387.089876] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11387.091711] LustreError: 340403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11387.342676] Lustre: server umount lustre-MDT0001 complete [11401.261957] Lustre: server umount lustre-OST0000 complete [11415.451788] Lustre: server umount lustre-OST0001 complete [11420.929490] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11423.000938] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11425.928174] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:06:49 (1787803609) [11448.718246] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [11459.279055] loop0: detected capacity change from 0 to 1099511627776 [11459.805426] SGI XFS with ACLs, security attributes, quota, no debug enabled [11459.812184] *** VALIDATE xfs *** [11459.818988] XFS (loop0): Mounting V5 Filesystem [11459.852178] XFS (loop0): Ending clean mount [11461.344457] loop1: detected capacity change from 0 to 18691697672192 [11565.596108] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [11577.502383] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [11615.298358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11615.745467] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11615.782774] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11615.911971] Lustre: lustre-MDT0000: new disk, initializing [11615.998065] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11616.001491] Lustre: Skipped 1 previous similar message [11616.012590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11621.094367] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11633.523366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11633.887589] Lustre: 348572:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11633.938055] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11633.952491] Lustre: Skipped 1 previous similar message [11634.058542] Lustre: lustre-MDT0001: new disk, initializing [11634.225204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11634.234211] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11639.183866] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11650.378389] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11657.494646] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11666.149472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11666.547743] Lustre: lustre-OST0000: new disk, initializing [11666.552204] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11666.557491] Lustre: 350546:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11668.519568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11668.527734] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11668.620709] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11673.479968] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11684.084522] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11688.766382] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11689.011251] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11693.070537] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11693.233791] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11699.171489] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11699.172027] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11699.181051] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11699.181059] Lustre: Skipped 4 previous similar messages [11699.192849] Lustre: Skipped 4 previous similar messages [11701.309297] Lustre: server umount lustre-OST0000 complete [11705.831235] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11705.843889] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11705.872301] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11705.875910] Lustre: Skipped 1 previous similar message [11711.499697] Lustre: server umount lustre-MDT0000 complete [11714.535113] LustreError: 350136:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11714.559610] LustreError: 350136:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [11715.472662] LustreError: 348564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803901 with bad export cookie 4612843332795412506 [11715.475298] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11715.481409] LustreError: 348564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11715.835821] Lustre: server umount lustre-MDT0001 complete [11726.371641] XFS (loop0): Unmounting Filesystem [11730.784801] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:11:55 (1787803915) [11738.775561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11739.314854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11739.319384] Lustre: Skipped 2 previous similar messages [11744.282806] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11754.765743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11755.013111] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [11755.024401] Lustre: Skipped 2 previous similar messages [11759.914937] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11766.992283] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11774.490883] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11785.067097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11796.559674] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11807.088474] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11825.633312] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11825.649213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11825.655837] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11825.670947] Lustre: Skipped 2 previous similar messages [11825.678737] Lustre: Skipped 1 previous similar message [11828.384529] Lustre: server umount lustre-OST0000 complete [11838.829091] Lustre: server umount lustre-MDT0000 complete [11843.141284] LustreError: 353083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787804028 with bad export cookie 4612843332795413388 [11843.153666] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11843.154489] LustreError: 353083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11843.574485] Lustre: server umount lustre-MDT0001 complete [11858.126676] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [11861.909530] Key type lgssc unregistered [11862.385218] LNet: 356735:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11862.409934] LNetError: 356735:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11862.442940] LNet: Removed LNI 192.168.206.116@tcp [11863.882309] Key type .llcrypt unregistered [11863.884262] Key type ._llcrypt unregistered [11895.943633] Key type ._llcrypt registered [11895.945671] Key type .llcrypt registered [11896.021726] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [11896.976812] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11896.997946] alg: No test for adler32 (adler32-zlib) [11898.213510] Lustre: Lustre: Build Version: 2.17.57_84_ga3423c7 [11898.608729] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [11900.327170] Key type lgssc registered [11902.754966] Lustre: Echo OBD driver; http://www.lustre.org/ [11912.973678] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11913.015142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11914.806710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11919.661078] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11924.967492] LustreError: 358301:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11924.982432] LustreError: 358301:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11929.497915] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11929.771445] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11934.483197] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11941.646459] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11948.435821] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11955.016471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11955.497073] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11963.163520] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11973.635985] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11985.378428] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11985.388533] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11985.426774] Lustre: Skipped 1 previous similar message [11986.775893] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [11986.790591] Lustre: Skipped 1 previous similar message [11989.655586] Lustre: server umount lustre-MDT0001 complete [11990.513144] LustreError: 359029:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11990.526066] LustreError: 359029:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11991.865709] LustreError: 358298:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11995.626286] LustreError: 358302:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11995.644041] LustreError: 358302:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12000.740101] LustreError: 358301:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12000.758959] LustreError: 358301:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12002.423652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12002.602172] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12002.609151] Lustre: Found index 1 for lustre-MDT0001, updating log [12002.652582] Lustre: 358290:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12003.044973] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12008.441858] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12008.921108] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [12009.480548] LustreError: 361771:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12009.485447] LustreError: 361771:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12009.491467] LustreError: 358293:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12009.602114] LustreError: 361778:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.116@tcp: cfg command failed: rc = -17 [12009.610034] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12009.610034] [12314.426393] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [12315.624251] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12315.625450] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12319.517232] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [12319.521616] Lustre: Skipped 2 previous similar messages [12324.641611] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [12324.648154] Lustre: Skipped 2 previous similar messages [12327.391172] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12327.527246] Lustre: server umount lustre-MDT0001 complete [12329.755771] LustreError: 358297:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12329.778630] LustreError: 358297:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12335.888043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12335.953206] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12335.956122] Lustre: Found index 1 for lustre-MDT0001, updating log [12335.982717] Lustre: 358291:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12336.096289] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12336.102532] Lustre: Skipped 3 previous similar messages [12336.291739] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12339.497480] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [12341.738219] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12342.773755] LustreError: 370536:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12342.779709] LustreError: 370536:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12342.787958] LustreError: 358293:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12342.805118] LustreError: 358293:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12342.841148] LustreError: 370539:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12342.847724] LustreError: 370539:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12342.855156] LustreError: 370539:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.116@tcp: cfg command failed: rc = -17 [12342.867767] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.116@tcp [12342.867767] [12647.216610] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [12648.937715] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12648.947307] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12648.964648] Lustre: Skipped 1 previous similar message [12650.456447] Lustre: server umount lustre-MDT0001 complete [12652.329899] LustreError: 358297:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12652.353329] LustreError: 358297:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12660.330893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12660.437908] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12660.443706] Lustre: Found index 1 for lustre-MDT0001, updating log [12660.495817] Lustre: 358290:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12660.753895] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12665.366207] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [12665.854021] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12669.942789] LustreError: 380449:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12669.955649] LustreError: 380449:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12669.966226] LustreError: 358293:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12669.981425] LustreError: 358293:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12670.069512] LustreError: 380452:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12670.081810] LustreError: 380452:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.116@tcp: cfg command failed: rc = -114 [12670.097588] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12670.097588] [12988.388423] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12988.390035] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12988.397775] Lustre: Skipped 2 previous similar messages [12988.400689] Lustre: Skipped 2 previous similar messages [12988.406526] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12991.545908] Lustre: server umount lustre-MDT0000 complete [12993.505985] LustreError: 358298:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12993.517196] LustreError: 358298:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [12994.337565] LustreError: 369553:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787805180 with bad export cookie 4305480715484536484 [12994.340506] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12994.355235] LustreError: 369553:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12998.626933] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12998.639618] Lustre: Skipped 1 previous similar message [13000.709865] Lustre: server umount lustre-MDT0001 complete [13004.462964] Lustre: server umount lustre-OST0000 complete [13013.247744] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [13022.272487] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [13054.569791] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 00:33:59 (1787805239) [13061.081829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13061.329380] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13061.373436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13061.474515] Lustre: lustre-MDT0000: new disk, initializing [13061.566850] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13061.583842] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13065.846942] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [13075.811132] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13075.920807] Lustre: 391479:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13075.972518] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13075.975908] Lustre: Skipped 1 previous similar message [13076.054902] Lustre: lustre-MDT0001: new disk, initializing [13076.133204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13076.168118] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13076.186249] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13079.290261] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [13086.190869] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13090.914822] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13096.288392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13096.572171] Lustre: lustre-OST0000: new disk, initializing [13096.575478] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13096.581447] Lustre: 393447:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13096.654158] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13101.195194] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [13105.195804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13105.202901] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13105.280372] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13108.142734] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13115.367575] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13115.373177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13115.385446] Lustre: Skipped 2 previous similar messages [13115.389893] Lustre: Skipped 5 previous similar messages [13119.735780] Lustre: server umount lustre-MDT0000 complete [13120.485737] LustreError: 391487:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13120.507256] LustreError: 391487:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [13122.451322] LustreError: 391472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787805308 with bad export cookie 4305480715484538437 [13122.456086] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13122.465027] LustreError: 391472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13122.702689] Lustre: server umount lustre-MDT0001 complete [13136.183740] Lustre: server umount lustre-OST0000 complete [13154.215200] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [13161.877532] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [13182.065987] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12932 sec ========= 00:36:06 (1787805366) [13183.348271] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 00:36:08 (1787805368) === [13185.460196] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 00:36:10 (1787805370) === [13210.727202] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [13212.724452] Key type lgssc unregistered [13212.960475] LNet: 399804:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13212.971440] LNetError: 399804:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13212.988303] LNet: Removed LNI 192.168.206.116@tcp [13213.602667] Key type .llcrypt unregistered [13213.605156] Key type ._llcrypt unregistered