[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 496806450 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001008] APIC: Switch to symmetric I/O mode setup [ 0.002365] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004014] kvm-guest: setup PV IPIs [ 0.007585] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009011] pid_max: default: 32768 minimum: 301 [ 0.010136] LSM: Security Framework initializing [ 0.011067] Yama: becoming mindful. [ 0.012038] SELinux: Initializing. [ 0.014060] *** VALIDATE selinux *** [ 0.022123] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026611] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027150] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028107] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030053] *** VALIDATE tmpfs *** [ 0.031451] *** VALIDATE proc *** [ 0.032265] *** VALIDATE cgroup *** [ 0.033010] *** VALIDATE cgroup2 *** [ 0.034291] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035148] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036012] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037027] Spectre V2 : User space: Vulnerable [ 0.038009] Speculative Store Bypass: Vulnerable [ 0.041493] debug: unmapping init [mem 0xffffffff91459000-0xffffffff91460fff] [ 0.043181] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044705] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045026] ... version: 2 [ 0.046016] ... bit width: 48 [ 0.047009] ... generic registers: 4 [ 0.048013] ... value mask: 0000ffffffffffff [ 0.049013] ... max period: 00007fffffffffff [ 0.050018] ... fixed-purpose events: 3 [ 0.051014] ... event mask: 000000070000000f [ 0.052326] rcu: Hierarchical SRCU implementation. [ 0.054410] smp: Bringing up secondary CPUs ... [ 0.055583] x86: Booting SMP configuration: [ 0.056027] .... node #0, CPUs: #1 #2 #3 [ 0.060076] smp: Brought up 1 node, 4 CPUs [ 0.062015] smpboot: Max logical packages: 1 [ 0.063013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.146175] node 0 deferred pages initialised in 80ms [ 0.149158] devtmpfs: initialized [ 0.150220] x86/mm: Memory block size: 128MB [ 0.152806] gcov: version magic: 0x41383552 [ 0.154402] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.159099] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.162342] pinctrl core: initialized pinctrl subsystem [ 0.164164] [ 0.164826] ************************************************************* [ 0.167011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.169010] ** ** [ 0.172010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.174011] ** ** [ 0.177012] ** This means that this kernel is built to expose internal ** [ 0.179012] ** IOMMU data structures, which may compromise security on ** [ 0.180008] ** your system. ** [ 0.182012] ** ** [ 0.185012] ** If you see this message and you are not debugging the ** [ 0.187010] ** kernel, report this immediately to your vendor! ** [ 0.189013] ** ** [ 0.192012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.194011] ************************************************************* [ 0.197667] NET: Registered protocol family 16 [ 0.199431] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.202062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.205060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.209101] cpuidle: using governor menu [ 0.210600] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.213602] PCI: Using configuration type 1 for base access [ 0.216122] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.226431] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.229085] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.232214] cryptd: max_cpu_qlen set to 1000 [ 0.235225] ACPI: Added _OSI(Module Device) [ 0.237017] ACPI: Added _OSI(Processor Device) [ 0.238014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.240015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.245914] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.252352] ACPI: Interpreter enabled [ 0.253000] ACPI: PM: (supports S0 S3 S4 S5) [ 0.256018] ACPI: Using IOAPIC for interrupt routing [ 0.258097] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.262400] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.274608] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.277045] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.280020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.284082] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.289364] acpiphp: Slot [2] registered [ 0.290150] acpiphp: Slot [5] registered [ 0.292169] acpiphp: Slot [6] registered [ 0.294136] acpiphp: Slot [7] registered [ 0.295121] acpiphp: Slot [8] registered [ 0.297125] acpiphp: Slot [9] registered [ 0.298129] acpiphp: Slot [10] registered [ 0.300128] acpiphp: Slot [3] registered [ 0.301088] acpiphp: Slot [4] registered [ 0.303129] acpiphp: Slot [11] registered [ 0.305095] acpiphp: Slot [12] registered [ 0.306084] acpiphp: Slot [13] registered [ 0.308123] acpiphp: Slot [14] registered [ 0.309124] acpiphp: Slot [15] registered [ 0.311092] acpiphp: Slot [16] registered [ 0.312110] acpiphp: Slot [17] registered [ 0.314082] acpiphp: Slot [18] registered [ 0.315102] acpiphp: Slot [19] registered [ 0.317108] acpiphp: Slot [20] registered [ 0.319092] acpiphp: Slot [21] registered [ 0.320096] acpiphp: Slot [22] registered [ 0.322159] acpiphp: Slot [23] registered [ 0.324090] acpiphp: Slot [24] registered [ 0.325100] acpiphp: Slot [25] registered [ 0.327115] acpiphp: Slot [26] registered [ 0.328093] acpiphp: Slot [27] registered [ 0.330114] acpiphp: Slot [28] registered [ 0.332133] acpiphp: Slot [29] registered [ 0.333085] acpiphp: Slot [30] registered [ 0.335100] acpiphp: Slot [31] registered [ 0.336079] PCI host bridge to bus 0000:00 [ 0.338018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.341022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.343022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.346031] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.349030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.351023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.353172] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.356025] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.359259] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.369793] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.375026] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.377017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.380022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.382017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.385287] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.387571] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.390041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.392697] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.399017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.411014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.417014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.422000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.436027] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.448015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.466019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.477123] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.490016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.497015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.512020] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.526014] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.534016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.540016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.555014] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.564477] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.573020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.592027] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.612018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.622747] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.636016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.647016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.664018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.678733] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.685016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.692015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.708018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.719071] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.722351] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.724303] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.726327] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.728221] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.735028] iommu: Default domain type: Passthrough [ 0.736533] SCSI subsystem initialized [ 0.738152] ACPI: bus type USB registered [ 0.739101] usbcore: registered new interface driver usbfs [ 0.742099] usbcore: registered new interface driver hub [ 0.744089] usbcore: registered new device driver usb [ 0.746174] pps_core: LinuxPPS API ver. 1 registered [ 0.748012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.751062] PTP clock support registered [ 0.755081] EDAC MC: Ver: 3.0.0 [ 0.758164] PCI: Using ACPI for IRQ routing [ 0.759861] NetLabel: Initializing [ 0.762014] NetLabel: domain hash size = 128 [ 0.763011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.765088] NetLabel: unlabeled traffic allowed by default [ 0.768100] vgaarb: loaded [ 0.769247] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.771016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.779000] clocksource: Switched to clocksource kvm-clock [ 0.893050] VFS: Disk quotas dquot_6.6.0 [ 0.894600] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.897172] *** VALIDATE ramfs *** [ 0.898575] *** VALIDATE hugetlbfs *** [ 0.900309] pnp: PnP ACPI init [ 0.902789] pnp: PnP ACPI: found 6 devices [ 0.922415] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.926272] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.928918] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.931403] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.934285] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.937064] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.940513] NET: Registered protocol family 2 [ 0.943161] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.948401] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.952335] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.957573] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.961270] TCP: Hash tables configured (established 65536 bind 65536) [ 0.964336] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.967459] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.969905] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.972349] NET: Registered protocol family 1 [ 0.975581] RPC: Registered named UNIX socket transport module. [ 0.977726] RPC: Registered udp transport module. [ 0.979730] RPC: Registered tcp transport module. [ 0.981506] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.984112] NET: Registered protocol family 44 [ 0.985990] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.988550] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.990906] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.993428] PCI: CLS 0 bytes, default 64 [ 0.995219] Unpacking initramfs... [ 2.278567] debug: unmapping init [mem 0xffff92d43cc54000-0xffff92d43ffbffff] [ 2.281224] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.282771] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.284901] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.703394] Initialise system trusted keyrings [ 2.704993] Key type blacklist registered [ 2.706919] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.713511] zbud: loaded [ 2.716804] *** VALIDATE nfs *** [ 2.717552] *** VALIDATE nfs4 *** [ 2.719015] pstore: using deflate compression [ 2.721781] Platform Keyring initialized [ 2.801467] NET: Registered protocol family 38 [ 2.802901] Key type asymmetric registered [ 2.804548] Asymmetric key parser 'x509' registered [ 2.806344] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.808462] io scheduler mq-deadline registered [ 2.809716] io scheduler kyber registered [ 2.810704] io scheduler bfq registered [ 2.811938] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.813932] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.816297] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.818102] ACPI: Power Button [PWRF] [ 2.822281] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.828946] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.837137] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.842660] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.852719] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.877731] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.907392] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.912494] Non-volatile memory driver v1.3 [ 2.914409] Linux agpgart interface v0.103 [ 2.941105] virtio_blk virtio1: [vda] 146680 512-byte logical blocks (75.1 MB/71.6 MiB) [ 2.943619] vda: detected capacity change from 0 to 75100160 [ 2.958545] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.961763] vdb: detected capacity change from 0 to 1073741824 [ 2.980190] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.982424] vdc: detected capacity change from 0 to 2621440000 [ 2.995075] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.997447] vdd: detected capacity change from 0 to 2621440000 [ 3.012913] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.015382] vde: detected capacity change from 0 to 4294967296 [ 3.031297] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.034548] vdf: detected capacity change from 0 to 4294967296 [ 3.041674] libphy: Fixed MDIO Bus: probed [ 3.047922] usbcore: registered new interface driver usbserial_generic [ 3.050805] usbserial: USB Serial support registered for generic [ 3.053375] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.057929] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.059889] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.062173] mousedev: PS/2 mouse device common for all mice [ 3.065743] rtc_cmos 00:05: RTC can wake from S4 [ 3.068602] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.072434] rtc_cmos 00:05: registered as rtc0 [ 3.075347] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.075410] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.078490] intel_pstate: CPU model not supported [ 3.082830] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.088639] hid: raw HID events driver (C) Jiri Kosina [ 3.090780] usbcore: registered new interface driver usbhid [ 3.092934] usbhid: USB HID core driver [ 3.094557] drop_monitor: Initializing network drop monitor service [ 3.097184] Initializing XFRM netlink socket [ 3.099372] NET: Registered protocol family 10 [ 3.102370] Segment Routing with IPv6 [ 3.104303] NET: Registered protocol family 17 [ 3.106569] mpls_gso: MPLS GSO support [ 3.111657] RAS: Correctable Errors collector initialized. [ 3.114029] AVX version of gcm_enc/dec engaged. [ 3.116216] AES CTR mode by8 optimization enabled [ 3.193875] sched_clock: Marking stable (3193843595, 0)->(4115543127, -921699532) [ 3.197716] registered taskstats version 1 [ 3.199712] Loading compiled-in X.509 certificates [ 3.201791] zswap: loaded using pool lzo/zbud [ 3.227183] Key type big_key registered [ 3.242080] Key type encrypted registered [ 3.243467] ima: No TPM chip found, activating TPM-bypass! [ 3.245509] ima: Allocated hash algorithm: sha1 [ 3.247182] ima: No architecture policies found [ 3.249126] evm: Initialising EVM extended attributes: [ 3.251042] evm: security.selinux [ 3.252218] evm: security.ima [ 3.253320] evm: security.capability [ 3.254705] evm: HMAC attrs: 0x1 [ 3.257188] rtc_cmos 00:05: setting system clock to 2026-09-08 02:03:54 UTC (1788833034) [ 3.263967] debug: unmapping init [mem 0xffffffff92403000-0xffffffff925fffff] [ 3.267028] debug: unmapping init [mem 0xffffffff91182000-0xffffffff91458fff] [ 3.276120] Write protecting the kernel read-only data: 28672k [ 3.279442] debug: unmapping init [mem 0xffffffff8f803000-0xffffffff8f9fffff] [ 3.282396] debug: unmapping init [mem 0xffffffff90114000-0xffffffff901fffff] [ 3.322371] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.330603] systemd[1]: Detected virtualization kvm. [ 3.332883] systemd[1]: Detected architecture x86-64. [ 3.335152] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.361897] systemd[1]: No hostname configured. [ 3.364037] systemd[1]: Set hostname to . [ 3.366332] random: systemd: uninitialized urandom read (16 bytes read) [ 3.369267] systemd[1]: Initializing machine ID from random generator. [ 3.427076] random: ln: uninitialized urandom read (6 bytes read) [ 3.511899] random: systemd: uninitialized urandom read (16 bytes read) [ 3.514470] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.518625] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.522666] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Sockets. Starting Setup Virtual Console... Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.194490] device-mapper: uevent: version 1.0.3 [ 4.196933] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.970968] virtio_net virtio0 ens2: renamed from eth0 [ 4.989980] random: fast init done [ 5.057597] scsi host0: ata_piix [ 5.078195] scsi host1: ata_piix [ 5.079952] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.082467] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.646773] dracut-initqueue[596]: RTNETLINK answers: File exists [ 9.759766] random: crng init done [ 9.761382] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.303322] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.513635] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.741294] SELinux: Disabled at runtime. [ 11.799748] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.806709] systemd[1]: Detected virtualization kvm. [ 11.808294] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.339582] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.342829] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.348185] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.351817] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.354631] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.361804] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.365385] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ 12.441962] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Initrd Root File System. Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... Starting Apply Kernel Variables... Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Paths. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 12.853062] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.183808] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.193735] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.352735] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.361510] EDAC sbridge: Ver: 1.1.2 [ 15.125612] Key type dns_resolver registered [ 15.424814] NFS: Registering the id_resolver key type [ 15.427125] Key type id_resolver registered [ 15.428745] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg255-server login: [ 41.591353] libcfs: loading out-of-tree module taints kernel. [ 41.613813] Key type ._llcrypt registered [ 41.615664] Key type .llcrypt registered [ 41.671236] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 53.175896] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 55.265814] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 55.307760] alg: No test for adler32 (adler32-zlib) [ 55.690016] hrtimer: interrupt took 2990129 ns [ 57.146535] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 58.447539] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 60.480171] Key type lgssc registered [ 62.627112] Lustre: Echo OBD driver; http://www.lustre.org/ [ 87.112826] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 137.303419] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 151.635421] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 151.662542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 152.963113] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 153.006995] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 153.111985] Lustre: lustre-MDT0000: new disk, initializing [ 153.235419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 153.265947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 158.186779] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 172.621627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 172.744191] Lustre: 6496:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 172.783504] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 172.786743] Lustre: Skipped 1 previous similar message [ 172.883729] Lustre: lustre-MDT0001: new disk, initializing [ 173.013739] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 173.056103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 173.061149] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 177.450895] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 183.921950] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 195.453821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 196.052918] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 196.069953] Lustre: lustre-OST0000: new disk, initializing [ 196.076170] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 196.082457] Lustre: 8435:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 196.167265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 201.206180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 201.241421] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 201.332228] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 204.539211] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 222.912036] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 223.068363] Lustre: lustre-OST0001: new disk, initializing [ 223.073263] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 223.077569] Lustre: 9508:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 223.143857] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 231.304076] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.590538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 233.598961] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 233.713988] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 245.312636] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 254.019811] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 261.023118] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing check_logdir /tmp/testlogs/ [ 269.223661] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing yml_node [ 275.132420] Lustre: DEBUG MARKER: Client: 2.17.57.106 [ 279.509784] Lustre: DEBUG MARKER: MDS: 2.17.57.106 [ 283.415609] Lustre: DEBUG MARKER: OSS: 2.17.57.106 [ 285.499716] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Sep 7 22:08:34 EDT 2026 [ 306.397465] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 307.765723] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 326.115164] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 326.116552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 326.135877] Lustre: Skipped 1 previous similar message [ 326.135956] Lustre: Skipped 1 previous similar message [ 330.869545] Lustre: server umount lustre-MDT0000 complete [ 334.354325] LustreError: 7444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788833365 with bad export cookie 5821265234055189366 [ 334.355615] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 334.361940] LustreError: 7444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 336.353739] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 336.356177] LustreError: 6502:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 336.369886] Lustre: Skipped 2 previous similar messages [ 336.382148] LustreError: 6502:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 336.385093] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 336.404514] Lustre: Skipped 4 previous similar messages [ 339.362904] LustreError: 6800:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 339.367635] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 339.395451] LustreError: 6800:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 339.416206] Lustre: Skipped 1 previous similar message [ 340.955186] Lustre: server umount lustre-MDT0001 complete [ 345.344612] Lustre: server umount lustre-OST0000 complete [ 349.930824] Lustre: server umount lustre-OST0001 complete [ 356.818090] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 367.810099] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 406.851866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 407.091461] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 407.123519] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 407.236634] Lustre: lustre-MDT0000: new disk, initializing [ 407.304031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 407.319491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 412.573175] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 423.575079] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 423.680921] Lustre: 15062:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 423.741563] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 423.748671] Lustre: Skipped 1 previous similar message [ 423.848863] Lustre: lustre-MDT0001: new disk, initializing [ 423.993900] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 424.059892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 424.080134] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 428.674201] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 439.557955] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 445.982858] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 454.447401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 454.699369] Lustre: lustre-OST0000: new disk, initializing [ 454.703989] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 454.712494] Lustre: 17033:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 454.851687] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 456.443589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 456.449784] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 456.575748] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 462.093099] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 474.003771] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 480.215452] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 480.562255] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 486.446638] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 486.673994] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 492.517983] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 492.532725] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 492.535202] Lustre: Skipped 1 previous similar message [ 492.536831] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 492.571413] LustreError: Skipped 1 previous similar message [ 502.755650] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 502.769057] Lustre: Skipped 3 previous similar messages [ 503.265580] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 503.567973] Lustre: server umount lustre-OST0000 complete [ 507.875254] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 507.886670] Lustre: Skipped 2 previous similar messages [ 522.208140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 522.389581] Lustre: server umount lustre-MDT0000 complete [ 523.233466] LustreError: 15071:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 523.244255] LustreError: 15071:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 526.152687] LustreError: 15859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788833557 with bad export cookie 5821265234055190899 [ 526.156028] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 526.159786] LustreError: 15859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 526.354685] Lustre: server umount lustre-MDT0001 complete [ 532.366738] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 22:12:41 (1788833561) [ 540.161983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 540.910428] LustreError: 19302:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.093950] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 546.025313] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 546.288955] LustreError: 19302:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 546.324339] LustreError: 19302:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 555.983955] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 561.390799] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 571.660599] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 578.982530] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 587.284570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 595.439497] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 603.221581] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 617.837513] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 627.684876] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 627.695577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 627.701751] Lustre: Skipped 8 previous similar messages [ 631.101715] Lustre: server umount lustre-MDT0000 complete [ 632.806227] LustreError: 19303:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 632.833451] LustreError: 19303:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 635.951728] LustreError: 19282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788833667 with bad export cookie 5821265234055191781 [ 635.966336] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 635.978600] LustreError: 19282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 636.704285] Lustre: server umount lustre-MDT0001 complete [ 651.940680] Lustre: server umount lustre-OST0000 complete [ 675.685296] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 686.769341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 687.162297] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 687.269062] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 687.276727] Lustre: Skipped 2 previous similar messages [ 692.525728] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 701.836856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 707.289704] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 710.582384] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 717.885070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 725.720652] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 733.711834] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 733.846910] Lustre: lustre-OST0001: new disk, initializing [ 733.850194] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 733.854572] Lustre: 26024:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 735.828402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 735.842060] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 735.997516] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 740.523907] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 751.181341] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 755.319832] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 768.993377] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 769.007618] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 769.024233] Lustre: Skipped 2 previous similar messages [ 769.033644] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 774.619950] Lustre: server umount lustre-MDT0000 complete [ 777.192684] LustreError: 23705:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 777.215334] LustreError: 23705:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 778.183222] LustreError: 23691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788833809 with bad export cookie 5821265234055192628 [ 778.193482] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 778.200747] LustreError: 23691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 778.499758] Lustre: server umount lustre-MDT0001 complete [ 792.077828] Lustre: server umount lustre-OST0000 complete [ 806.258830] Lustre: server umount lustre-OST0001 complete [ 825.962907] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 837.377515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 837.864755] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 837.872119] Lustre: Skipped 3 previous similar messages [ 843.235490] LustreError: 29046:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 843.255138] LustreError: 29046:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 843.416748] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 853.189880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 853.474269] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 853.483338] Lustre: Skipped 4 previous similar messages [ 858.837036] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 861.720402] Lustre: 30193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 868.567115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 875.398512] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 883.730647] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 893.167490] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 903.174163] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 907.167772] Lustre: 32075:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 914.920885] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 914.931901] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 914.933891] Lustre: Skipped 5 previous similar messages [ 920.488667] Lustre: server umount lustre-MDT0000 complete [ 924.444952] LustreError: 30195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788833955 with bad export cookie 5821265234055193923 [ 924.469550] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 925.036074] Lustre: server umount lustre-MDT0001 complete [ 937.982908] Lustre: server umount lustre-OST0000 complete [ 941.536178] Lustre: 3625:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788833956/real 1788833956] req@ffff92d3867a1c00 x1875727445440000/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788833972 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 943.904178] Lustre: server umount lustre-OST0001 complete [ 954.502333] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 22:19:43 (1788833983) [ 987.340496] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1000.532553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1000.944137] LustreError: 34877:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1000.965048] LustreError: 34877:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 1006.820210] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1018.457494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1025.888519] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1029.712179] Lustre: 36027:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1038.867776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1048.121658] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1058.901575] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1066.703298] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1074.802958] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1078.921218] Lustre: 37907:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1090.026606] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1090.042542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1090.073315] Lustre: Skipped 5 previous similar messages [ 1090.099039] Lustre: Skipped 11 previous similar messages [ 1093.387062] Lustre: server umount lustre-MDT0000 complete [ 1098.520192] LustreError: 34859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788834129 with bad export cookie 5821265234055195029 [ 1098.524663] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1098.534095] LustreError: 34859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1098.935940] Lustre: server umount lustre-MDT0001 complete [ 1114.552835] Lustre: server umount lustre-OST0000 complete [ 1116.384294] Lustre: 3626:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788834131/real 1788834131] req@ffff92d4b7f27480 x1875727445495296/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788834147 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1119.197439] Lustre: server umount lustre-OST0001 complete [ 1128.911407] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 22:22:37 (1788834157) [ 1147.953952] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1159.444348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1159.925878] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1159.939439] Lustre: Skipped 7 previous similar messages [ 1165.296496] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1175.565350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1181.366474] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1184.608425] Lustre: 41196:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1191.627742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1199.786763] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1210.427701] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1218.569547] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1228.493306] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1233.288673] Lustre: 43073:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1234.532990] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1234.542876] Lustre: Skipped 1 previous similar message [ 1241.581789] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1241.603741] Lustre: Skipped 2 previous similar messages [ 1244.425625] Lustre: server umount lustre-MDT0000 complete [ 1248.795763] LustreError: 40800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788834280 with bad export cookie 5821265234055196135 [ 1248.801406] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1248.822726] LustreError: 40800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1255.344491] Lustre: server umount lustre-MDT0001 complete [ 1264.587356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1274.340590] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x50c949e35330f232 [ 1274.352661] Lustre: MGC192.168.202.155@tcp: Connection restored to 0@lo (at 0@lo) [ 1274.823587] LustreError: 41559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1274.855601] LustreError: 41559:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 32 previous similar messages [ 1280.043185] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1284.339695] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1291.775784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1297.910509] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1297.919146] Lustre: Skipped 1 previous similar message [ 1298.695129] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1309.059468] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1311.279715] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1317.761846] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1319.693414] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1328.101770] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1331.278288] Lustre: server umount lustre-MDT0000 complete [ 1335.230076] LustreError: 40800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788834366 with bad export cookie 5821265234055197234 [ 1335.236183] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1335.247655] LustreError: 40800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1335.637688] Lustre: server umount lustre-MDT0001 complete [ 1350.299989] Lustre: server umount lustre-OST0000 complete [ 1364.981754] Lustre: server umount lustre-OST0001 complete [ 1377.911389] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 22:26:47 (1788834407) [ 1396.625959] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1406.930731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1411.404560] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1420.908623] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1425.301966] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1428.778517] Lustre: 48838:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1438.029136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1445.929288] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1456.146570] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1462.528806] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1470.621498] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1475.747710] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1492.455070] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1492.469687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1492.493931] Lustre: Skipped 9 previous similar messages [ 1492.515438] Lustre: Skipped 17 previous similar messages [ 1494.858113] Lustre: server umount lustre-MDT0000 complete [ 1499.740593] LustreError: 47668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788834530 with bad export cookie 5821265234055198053 [ 1499.754174] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1499.756600] LustreError: 47668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1500.469301] Lustre: server umount lustre-MDT0001 complete [ 1515.045050] Lustre: server umount lustre-OST0000 complete [ 1517.857162] Lustre: 3626:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788834533/real 1788834533] req@ffff92d484dfb480 x1875727445648512/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788834549 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1520.368577] Lustre: server umount lustre-OST0001 complete [ 1532.742631] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 22:29:21 (1788834561) [ 1535.939819] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1538.471441] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 22:29:27 (1788834567) [ 1551.573621] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 1562.363039] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1587.733911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1588.026772] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1588.111841] Lustre: lustre-MDT0000: new disk, initializing [ 1588.193161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1593.041977] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1603.059432] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1611.433713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1611.811457] Lustre: lustre-OST0000: new disk, initializing [ 1611.815638] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1611.818913] Lustre: Skipped 1 previous similar message [ 1611.823076] Lustre: 54921:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1613.288864] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1613.301214] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1613.339176] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1619.183278] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1631.025377] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1633.516811] Lustre: DEBUG MARKER: create test files [ 1675.681641] Lustre: server umount lustre-OST0000 complete [ 1680.469729] Lustre: server umount lustre-MDT0000 complete [ 1691.517908] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1694.046806] Key type lgssc unregistered [ 1694.398045] LNet: 56554:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1694.410768] LNetError: 56554:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1694.430835] LNet: Removed LNI 192.168.202.155@tcp [ 1695.342185] Key type .llcrypt unregistered [ 1695.344547] Key type ._llcrypt unregistered [ 1730.158924] Key type ._llcrypt registered [ 1730.160341] Key type .llcrypt registered [ 1730.267494] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1731.249625] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1731.738216] alg: No test for adler32 (adler32-zlib) [ 1733.060206] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 1733.395951] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1735.123228] Key type lgssc registered [ 1736.529750] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1744.781279] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1744.808652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1746.310608] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1750.688488] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1757.930372] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1765.498115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1765.783193] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1770.945902] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1770.981666] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1771.098542] Lustre: 59293:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1771.110118] Lustre: 59293:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1771.119393] Lustre: 59293:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1771.123988] Lustre: 59293:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1771.132475] Lustre: 59293:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1771.137628] Lustre: 59293:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1780.510887] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1786.220555] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1792.979390] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1801.697910] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1801.708780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1806.818549] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1807.306933] Lustre: server umount lustre-OST0000 complete [ 1812.307389] Lustre: server umount lustre-MDT0000 complete [ 1826.746682] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1829.750730] Key type lgssc unregistered [ 1830.086423] LNet: 60900:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1830.092639] LNetError: 60900:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1830.125323] LNet: Removed LNI 192.168.202.155@tcp [ 1831.163127] Key type .llcrypt unregistered [ 1831.166643] Key type ._llcrypt unregistered [ 1865.565442] Key type ._llcrypt registered [ 1865.568847] Key type .llcrypt registered [ 1865.690551] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1867.712473] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1867.732338] alg: No test for adler32 (adler32-zlib) [ 1868.899544] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 1869.221931] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1870.992936] Key type lgssc registered [ 1872.356383] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1880.765825] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1880.815240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1882.275178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1886.738948] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1895.195176] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1901.990047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1902.369240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1907.499594] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1907.590029] Lustre: 63732:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1907.600838] Lustre: 63732:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1907.607911] Lustre: 63732:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1907.612451] Lustre: 63732:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1907.625159] Lustre: 63732:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1907.631694] Lustre: 63732:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1909.026528] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1917.267447] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1922.682169] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1933.283568] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1933.292543] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1933.303641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1936.644635] Lustre: server umount lustre-OST0000 complete [ 1941.173700] Lustre: server umount lustre-MDT0000 complete [ 1953.802497] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1957.084383] Key type lgssc unregistered [ 1957.532518] LNet: 65288:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1957.543071] LNetError: 65288:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1957.560903] LNet: Removed LNI 192.168.202.155@tcp [ 1958.660936] Key type .llcrypt unregistered [ 1958.663626] Key type ._llcrypt unregistered [ 1978.528317] Key type ._llcrypt registered [ 1978.530599] Key type .llcrypt registered [ 1978.652208] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 1995.429666] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1996.610216] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1996.666966] alg: No test for adler32 (adler32-zlib) [ 1997.817983] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 1998.150951] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1999.864210] Key type lgssc registered [ 2001.572917] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2042.782816] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2042.829106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2044.164725] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2044.182936] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2044.356182] Lustre: lustre-MDT0000: new disk, initializing [ 2044.533330] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2044.571472] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2049.913381] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2061.840275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2062.017896] Lustre: 68442:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2062.130228] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2062.145453] Lustre: Skipped 1 previous similar message [ 2062.350373] Lustre: lustre-MDT0001: new disk, initializing [ 2062.523331] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2062.603758] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2062.617613] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2067.900732] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2079.454560] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2087.436357] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2095.616102] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2096.224466] Lustre: lustre-OST0000: new disk, initializing [ 2096.231348] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2096.241452] Lustre: 70412:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2096.410161] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2104.370538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2104.375263] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2104.495191] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2105.890433] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2118.162705] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2123.183400] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2123.383567] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2129.690991] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2130.019440] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2135.010109] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2135.016197] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2135.027940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2135.524752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2135.560750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2138.866976] Lustre: server umount lustre-OST0000 complete [ 2144.741949] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2144.756615] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2144.792415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2148.923483] Lustre: server umount lustre-MDT0000 complete [ 2152.859740] LustreError: 68434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788835184 with bad export cookie 4673361923311456281 [ 2152.863268] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2152.867131] LustreError: 68434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2153.219537] Lustre: server umount lustre-MDT0001 complete [ 2162.747435] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:39:51 (1788835191) [ 2173.792496] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2218.196457] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 2228.531621] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2273.729188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2274.064325] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2274.098082] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2274.206655] Lustre: lustre-MDT0000: new disk, initializing [ 2274.351174] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2274.384727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2279.888431] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2291.101950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2291.243469] Lustre: 76432:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2291.306222] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2291.310616] Lustre: Skipped 1 previous similar message [ 2291.404421] Lustre: lustre-MDT0001: new disk, initializing [ 2291.534912] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2291.601509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2291.630566] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2296.555295] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2307.320647] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2314.962368] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2324.385532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2324.875246] Lustre: lustre-OST0000: new disk, initializing [ 2324.880027] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2324.887365] Lustre: 78401:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2324.994715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2331.196918] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2331.209399] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2331.386800] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2332.525137] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2344.854537] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2351.111029] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2351.438370] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2357.543374] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2357.917184] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2362.339083] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2362.340765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2362.349269] LustreError: Skipped 1 previous similar message [ 2362.351325] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2362.351333] Lustre: Skipped 2 previous similar messages [ 2362.386209] Lustre: Skipped 2 previous similar messages [ 2366.763895] Lustre: server umount lustre-OST0000 complete [ 2372.588805] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2372.615495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2372.625347] Lustre: Skipped 1 previous similar message [ 2372.648615] Lustre: Skipped 2 previous similar messages [ 2378.078043] Lustre: server umount lustre-MDT0000 complete [ 2381.793977] LustreError: 76442:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2381.816534] LustreError: 76442:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2382.538026] LustreError: 76423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788835413 with bad export cookie 4673361923311457156 [ 2382.548503] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2382.550986] LustreError: 76423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2383.018575] Lustre: server umount lustre-MDT0001 complete [ 2392.762673] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:43:41 (1788835421) [ 2399.982917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2400.326608] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2400.413381] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2405.860053] LustreError: 80817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2406.192566] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2409.957606] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2415.074168] LustreError: 80817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2415.438951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2421.515907] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2430.089219] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2436.478767] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2443.435512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2443.850862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2443.864954] Lustre: Skipped 1 previous similar message [ 2451.271569] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2460.199825] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2464.227031] Lustre: 83489:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 30b10fd4-b0f7-409f-ad46-346ef88cd295 at adminstrative request [ 2465.771650] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2465.790058] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2476.007050] Lustre: MGS: Client 30b10fd4-b0f7-409f-ad46-346ef88cd295 (at 0@lo) reconnecting [ 2476.029598] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x40db1de8a1d0d6fd to 0x40db1de8a1d0d99d [ 2476.060062] Lustre: MGC192.168.202.155@tcp: Connection restored to 0@lo (at 0@lo) [ 2505.069483] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2505.344449] Lustre: lustre-OST0001: new disk, initializing [ 2505.347996] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2505.355436] Lustre: 83883:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2511.630369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2511.650043] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2511.822730] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2514.424359] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2523.651833] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2532.336126] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2532.350399] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2532.353154] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2532.353161] Lustre: Skipped 2 previous similar messages [ 2532.388699] Lustre: Skipped 1 previous similar message [ 2533.787418] Lustre: server umount lustre-OST0001 complete [ 2537.443131] LustreError: 82637:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2537.467833] LustreError: 82637:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2542.562928] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2542.574199] LustreError: Skipped 2 previous similar messages [ 2544.374380] Lustre: server umount lustre-OST0000 complete [ 2548.713823] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2548.737235] Lustre: Skipped 2 previous similar messages [ 2548.746162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2548.754791] Lustre: Skipped 3 previous similar messages [ 2555.090776] Lustre: server umount lustre-MDT0000 complete [ 2557.934539] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2557.956274] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2559.531239] LustreError: 80796:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788835590 with bad export cookie 4673361923311458717 [ 2559.542737] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2559.547125] LustreError: 80796:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2560.138514] Lustre: server umount lustre-MDT0001 complete [ 2570.334357] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:46:39 (1788835599) [ 2572.244253] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2574.266464] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:46:43 (1788835603) [ 2576.127963] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2578.017904] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:46:47 (1788835607) [ 2579.675747] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2581.331143] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:46:50 (1788835610) [ 2583.072665] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2585.359774] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:46:54 (1788835614) [ 2587.842598] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2596.133477] loop: module loaded [ 2596.161862] loop0: detected capacity change from 0 to 4294967296 [ 2596.222492] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2598.174573] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2601.658116] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2605.964402] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2639.749309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2639.827036] LustreError: 88010:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2639.834543] LustreError: 88010:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2639.839896] LustreError: 88010:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2639.845819] LustreError: 88010:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2666.186619] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 2681.542431] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2728.065659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2728.414495] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2728.444610] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2728.506461] Lustre: lustre-MDT0000: new disk, initializing [ 2728.585364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2728.591641] Lustre: Skipped 1 previous similar message [ 2728.610421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2735.404956] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2748.220191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2748.469373] Lustre: 90901:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2748.828056] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2754.094104] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2765.161122] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2772.236603] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2779.766200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2780.002951] Lustre: 92869:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2787.429213] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2789.976066] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2790.144336] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2797.989853] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2805.900258] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2806.357267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2810.688925] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2810.846934] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2815.970151] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2815.978563] LustreError: Skipped 1 previous similar message [ 2815.980750] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2815.987578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2815.992589] Lustre: Skipped 2 previous similar messages [ 2816.002570] Lustre: Skipped 3 previous similar messages [ 2827.232616] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2827.456846] Lustre: server umount lustre-OST0000 complete [ 2835.938916] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2839.149495] Lustre: server umount lustre-MDT0000 complete [ 2841.574476] LustreError: 90913:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2841.605940] LustreError: 90913:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2843.970575] LustreError: 90894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788835875 with bad export cookie 4673361923311459333 [ 2843.979187] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2843.996471] LustreError: 90894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2844.022233] LustreError: 66661:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [ 2844.573295] Lustre: server umount lustre-MDT0001 complete [ 2855.002244] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:51:23 (1788835883) [ 2864.149344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2864.837192] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2864.845321] Lustre: Skipped 2 previous similar messages [ 2871.335746] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2882.607566] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2888.591233] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2898.263880] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2904.721314] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2912.927214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2920.851195] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2929.614779] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2935.738464] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2935.997554] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2941.137489] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2941.384517] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2948.752594] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2948.950447] Lustre: lustre-OST0001: new disk, initializing [ 2948.953287] Lustre: Skipped 2 previous similar messages [ 2948.958053] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2948.963338] Lustre: Skipped 3 previous similar messages [ 2948.972387] Lustre: 98657:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2956.241815] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2957.980329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2958.001748] Lustre: Skipped 2 previous similar messages [ 2958.009993] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2958.133724] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2966.863773] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2971.815194] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2972.125806] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2977.165256] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2977.370760] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3000.868043] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3029.306781] Lustre: Failing over lustre-MDT0000 [ 3029.767326] Lustre: server umount lustre-MDT0000 complete [ 3029.998334] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3030.010361] LustreError: 95281:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3030.020396] Lustre: Skipped 4 previous similar messages [ 3030.036328] LustreError: 95281:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 3051.499460] Lustre: 66662:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788836066/real 1788836066] req@00000000c4cd8679 x1875729479664000/t0(0) o400->MGC192.168.202.155@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788836082 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3051.531812] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3058.637385] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3058.639681] LDISKFS-fs (dm-0): recovery complete [ 3058.649308] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3060.896167] LustreError: 101478:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 3060.912804] LustreError: 101478:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000041399b2 x1875729479669120/t0(0) o250->MGC192.168.202.155@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1788836092 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3060.948072] LustreError: 101478:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 3061.828832] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.55@tcp (not set up) [ 3061.844606] Lustre: Skipped 8 previous similar messages [ 3062.241288] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3066.956436] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3066.957470] Lustre: 99433:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3066.972658] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3067.372399] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3067.461418] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3067.474527] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 3067.493562] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3067.808222] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3068.174323] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3068.195836] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3068.524099] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3068.527950] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3068.533973] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3068.548054] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 3 previous similar messages [ 3069.194040] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3069.212234] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3069.539670] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3069.552338] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3070.573323] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3070.591208] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 3071.234151] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3071.242570] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 3071.584188] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3071.590765] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 3074.792293] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3074.823539] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 3075.489378] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3075.511422] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3075.832386] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3075.849206] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3082.909128] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3082.928759] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 3083.587133] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3083.600389] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3083.943537] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3083.953559] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3098.978659] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3098.990181] Lustre: 101509:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 49 previous similar messages [ 3099.647192] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3099.661504] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 3099.984217] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3099.993684] LustreError: 101509:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 3126.048130] Lustre: 101509:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3126.052240] Lustre: 101509:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3126.057814] Lustre: 101509:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client c374b697-eeb9-4ace-8994-4eb00453156a@192.168.202.55@tcp [ 3126.070573] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3126.074728] Lustre: 101509:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000728931ac x1875729461617152/t0(8589934783) o101->c374b697-eeb9-4ace-8994-4eb00453156a@192.168.202.55@tcp:322/0 lens 592/0 e 0 to 0 dl 1788836167 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3126.099888] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3126.116388] Lustre: lustre-MDT0000: Denying connection for new client c374b697-eeb9-4ace-8994-4eb00453156a (at 192.168.202.55@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3126.129671] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3126.185451] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3126.187353] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3126.188034] Lustre: Skipped 2 previous similar messages [ 3126.221856] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 3126.226676] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3142.628782] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3146.036323] Lustre: server umount lustre-OST0000 complete [ 3159.528602] LustreError: 101888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3159.546252] LustreError: 101888:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 49 previous similar messages [ 3164.129288] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3164.682388] Lustre: server umount lustre-OST0001 complete [ 3169.764677] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3169.776949] Lustre: Skipped 5 previous similar messages [ 3175.867905] Lustre: server umount lustre-MDT0000 complete [ 3180.453368] LustreError: 98656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788836211 with bad export cookie 4673361923311510545 [ 3180.461683] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3180.470674] LustreError: 98656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3180.931344] Lustre: server umount lustre-MDT0001 complete [ 3190.246912] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 22:56:59 (1788836219) [ 3198.941246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3198.947380] Lustre: *** cfs_fail_loc=197, val=0*** [ 3214.497977] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3219.616697] LustreError: 103488:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.155@tcp: failed processing log, type 4: rc = -110 [ 3224.770537] Lustre: *** cfs_fail_loc=197, val=0*** [ 3224.772641] Lustre: Skipped 65 previous similar messages [ 3245.345141] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3245.368828] Lustre: Skipped 4 previous similar messages [ 3251.152533] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3259.086962] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3261.369547] Lustre: server umount lustre-OST0000 complete [ 3270.638398] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:58:19 (1788836299) [ 3292.604213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3299.068808] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3309.172821] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3313.438982] LustreError: 105139:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3313.445296] LustreError: 105139:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3313.450233] LustreError: 105139:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3314.001672] Lustre: server umount lustre-MDT0000 complete [ 3353.166721] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3364.688314] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3416.592070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3417.086349] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3417.155737] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3417.369536] Lustre: lustre-MDT0000: new disk, initializing [ 3417.593298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3423.647576] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3436.618688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3436.741230] Lustre: 109317:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3436.933808] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3442.487886] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3454.736984] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3462.613987] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3470.947735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3471.349842] Lustre: 111299:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3476.724099] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3479.022497] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3490.873615] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3496.855780] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3497.216872] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3503.324738] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3503.577194] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3507.686606] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3507.687573] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3507.699766] LustreError: Skipped 4 previous similar messages [ 3507.701916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3507.701923] Lustre: Skipped 14 previous similar messages [ 3507.731031] Lustre: Skipped 2 previous similar messages [ 3512.150516] Lustre: server umount lustre-OST0000 complete [ 3531.233064] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3531.506181] Lustre: server umount lustre-MDT0000 complete [ 3533.284150] LustreError: 111765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3533.299364] LustreError: 111765:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 3536.551276] LustreError: 109310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788836567 with bad export cookie 4673361923311521192 [ 3536.563400] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3536.567399] LustreError: 109310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3536.990063] Lustre: server umount lustre-MDT0001 complete [ 3548.044787] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 23:02:57 (1788836577) [ 3558.975542] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 3562.419324] Key type lgssc unregistered [ 3562.766607] LNet: 113792:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3562.777042] LNetError: 113792:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3562.808262] LNet: Removed LNI 192.168.202.155@tcp [ 3563.783871] Key type .llcrypt unregistered [ 3563.786254] Key type ._llcrypt unregistered [ 3585.227687] Key type ._llcrypt registered [ 3585.231950] Key type .llcrypt registered [ 3585.349318] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3603.505200] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3604.731478] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3604.767256] alg: No test for adler32 (adler32-zlib) [ 3606.029921] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 3606.371294] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 3608.088207] Key type lgssc registered [ 3609.740490] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3657.661520] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3657.737907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3659.133593] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3659.188128] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3659.326511] Lustre: lustre-MDT0000: new disk, initializing [ 3659.442889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3659.472539] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3665.222974] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3678.478288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3678.820549] Lustre: lustre-OST0000: new disk, initializing [ 3678.825302] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3678.830523] Lustre: Skipped 1 previous similar message [ 3678.835681] Lustre: 117965:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3678.910940] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3686.346247] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3687.555435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3687.568888] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3687.639983] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3699.050412] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3699.282562] Lustre: lustre-OST0001: new disk, initializing [ 3699.287217] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3699.299159] Lustre: 118828:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3699.401532] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3705.994085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3706.005101] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3706.050389] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3706.611993] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3716.067900] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3716.071176] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3716.080140] Lustre: Skipped 1 previous similar message [ 3716.098670] Lustre: Skipped 1 previous similar message [ 3721.642648] Lustre: server umount lustre-MDT0000 complete [ 3728.072298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3732.736073] LustreError: 117154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788836763 with bad export cookie 11704952248158161397 [ 3732.745240] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3732.745909] LustreError: 117154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3732.833735] Lustre: server umount lustre-OST0000 complete [ 3738.193756] Lustre: server umount lustre-OST0001 complete [ 3750.377331] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 23:06:19 (1788836779) [ 3777.134234] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 23:06:46 (1788836806) [ 3787.273334] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3812.114528] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 23:07:21 (1788836841) [ 3858.893529] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3870.267259] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3908.490593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3908.788866] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3908.805864] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3908.855591] Lustre: lustre-MDT0000: new disk, initializing [ 3908.929804] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3908.942788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3914.233296] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3927.511818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3927.653744] Lustre: 126587:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3927.719431] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3927.723658] Lustre: Skipped 1 previous similar message [ 3927.828534] Lustre: lustre-MDT0001: new disk, initializing [ 3927.950818] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3928.004863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3928.018228] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3933.458749] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3943.514375] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3949.877948] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3957.828789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3958.244045] Lustre: lustre-OST0000: new disk, initializing [ 3958.249664] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3958.264624] Lustre: 128555:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3958.421707] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3966.609591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3966.630381] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3966.801137] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3966.871484] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3977.855606] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3983.095145] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3983.367855] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3988.143409] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3988.387498] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3992.545330] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3992.552362] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3992.559597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3992.589214] Lustre: Skipped 1 previous similar message [ 3997.674673] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3997.680188] Lustre: Skipped 2 previous similar messages [ 4002.790709] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4002.812794] Lustre: Skipped 1 previous similar message [ 4005.344149] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4005.599922] Lustre: server umount lustre-OST0000 complete [ 4013.029657] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4013.053671] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4013.062110] Lustre: Skipped 1 previous similar message [ 4016.532712] Lustre: server umount lustre-MDT0000 complete [ 4018.150515] LustreError: 126598:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4018.167147] LustreError: 126598:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 4020.472271] LustreError: 126579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837051 with bad export cookie 11704952248158162629 [ 4020.474609] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4020.481734] LustreError: 126579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4021.070989] Lustre: server umount lustre-MDT0001 complete [ 4031.708602] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 23:11:00 (1788837060) [ 4040.990209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4041.593702] LustreError: 130969:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4041.688333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4046.817453] LustreError: 130970:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4046.986193] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4050.924701] LustreError: 130969:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4055.896401] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4056.038341] LustreError: 130970:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4060.630207] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4068.744716] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4074.617557] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4082.743884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4083.066884] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4083.076756] Lustre: Skipped 1 previous similar message [ 4090.321595] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4098.710145] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4106.935536] Lustre: *** cfs_fail_loc=159, val=0*** [ 4106.941506] Lustre: Skipped 2 previous similar messages [ 4122.287937] Lustre: lustre-MDT0000: Client 72c960c5-3a1c-4a82-95ce-2d1c01797b9b (at 192.168.202.55@tcp) reconnecting [ 4125.160571] Lustre: *** cfs_fail_loc=159, val=0*** [ 4125.166378] Lustre: Skipped 4 previous similar messages [ 4141.674218] Lustre: lustre-MDT0000: Client 72c960c5-3a1c-4a82-95ce-2d1c01797b9b (at 192.168.202.55@tcp) reconnecting [ 4147.701682] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4147.702774] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4147.726322] Lustre: Skipped 1 previous similar message [ 4147.739449] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4147.743598] Lustre: Skipped 1 previous similar message [ 4154.001082] Lustre: server umount lustre-OST0000 complete [ 4157.927488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4157.941420] Lustre: Skipped 2 previous similar messages [ 4164.241899] Lustre: server umount lustre-MDT0000 complete [ 4168.160119] LustreError: 133568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837199 with bad export cookie 11704952248158163504 [ 4168.168188] LustreError: 133568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4168.170521] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4168.754294] Lustre: server umount lustre-MDT0001 complete [ 4180.867398] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4184.430712] Key type lgssc unregistered [ 4184.808820] LNet: 134749:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4184.822466] LNetError: 134749:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4184.838727] LNet: Removed LNI 192.168.202.155@tcp [ 4185.924254] Key type .llcrypt unregistered [ 4185.929412] Key type ._llcrypt unregistered [ 4212.972478] Key type ._llcrypt registered [ 4212.974219] Key type .llcrypt registered [ 4213.089178] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4214.296919] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4214.351686] alg: No test for adler32 (adler32-zlib) [ 4215.577058] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 4216.118618] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4217.971691] Key type lgssc registered [ 4219.319180] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4227.545042] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4227.576745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4229.275615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4234.746564] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4238.827563] LustreError: 136162:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4238.859942] LustreError: 136162:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4243.690265] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4244.052922] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4248.523994] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4256.523315] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4262.971616] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4270.582117] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4270.930834] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4274.034795] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4277.690189] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4285.756936] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4297.958618] Lustre: *** cfs_fail_loc=159, val=0*** [ 4313.201939] Lustre: lustre-MDT0000: Client bef4e0c1-689d-4bc6-8542-bebe48761122 (at 192.168.202.55@tcp) reconnecting [ 4315.598538] Lustre: *** cfs_fail_loc=159, val=0*** [ 4332.160418] Lustre: lustre-MDT0001: Client bef4e0c1-689d-4bc6-8542-bebe48761122 (at 192.168.202.55@tcp) reconnecting [ 4334.058018] Lustre: *** cfs_fail_loc=159, val=0*** [ 4334.066404] Lustre: Skipped 3 previous similar messages [ 4349.574490] Lustre: lustre-MDT0001: Client bef4e0c1-689d-4bc6-8542-bebe48761122 (at 192.168.202.55@tcp) reconnecting [ 4354.424045] Lustre: 139206:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4357.483473] Lustre: *** cfs_fail_loc=159, val=0*** [ 4357.486128] Lustre: Skipped 4 previous similar messages [ 4374.136420] Lustre: lustre-MDT0000: Client 759287ec-3b53-4c54-b63c-4acd24322862 (at 192.168.202.55@tcp) reconnecting [ 4375.385704] Lustre: *** cfs_fail_loc=159, val=0*** [ 4375.392508] Lustre: Skipped 15 previous similar messages [ 4392.070866] Lustre: lustre-MDT0000: Client 759287ec-3b53-4c54-b63c-4acd24322862 (at 192.168.202.55@tcp) reconnecting [ 4394.831684] Lustre: 139493:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4401.139879] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4401.141320] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4401.147451] LustreError: Skipped 3 previous similar messages [ 4401.184524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4402.146186] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4402.147188] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4402.148313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4402.149659] LustreError: Skipped 2 previous similar messages [ 4407.074996] Lustre: server umount lustre-OST0000 complete [ 4412.388047] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4412.393961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4412.399021] Lustre: Skipped 1 previous similar message [ 4412.417789] Lustre: Skipped 2 previous similar messages [ 4417.509535] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4417.522480] Lustre: Skipped 1 previous similar message [ 4417.927804] Lustre: server umount lustre-MDT0000 complete [ 4422.134814] LustreError: 136142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837453 with bad export cookie 9788571012831966747 [ 4422.152234] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4422.156017] LustreError: 136142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4422.573641] Lustre: server umount lustre-MDT0001 complete [ 4434.612749] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4437.746562] Key type lgssc unregistered [ 4438.052695] LNet: 140552:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4438.061893] LNetError: 140552:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4439.083743] LNet: Removed LNI 192.168.202.155@tcp [ 4439.934461] Key type .llcrypt unregistered [ 4439.935439] Key type ._llcrypt unregistered [ 4463.234493] Key type ._llcrypt registered [ 4463.235817] Key type .llcrypt registered [ 4463.345020] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4464.490571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4464.560417] alg: No test for adler32 (adler32-zlib) [ 4465.516159] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 4465.716566] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4467.384244] Key type lgssc registered [ 4468.473628] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4475.819656] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4475.845820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4477.247076] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4481.217790] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4487.657044] LustreError: 141962:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4487.681204] LustreError: 141962:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4489.829842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4490.397574] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4495.995616] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4504.343393] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4512.327094] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4521.723974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4522.271511] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4523.344870] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4529.623633] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4535.610366] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4537.064803] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4544.333163] Lustre: 144546:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4547.412451] Lustre: 144594:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4548.794093] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4549.393784] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4553.912624] Lustre: 144741:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4555.136927] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4556.367960] Lustre: 144840:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4561.398745] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4561.403429] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4561.417249] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4561.432560] Lustre: Skipped 1 previous similar message [ 4566.503540] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4566.506066] Lustre: Skipped 1 previous similar message [ 4567.759140] Lustre: server umount lustre-OST0000 complete [ 4576.752310] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4576.780813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4576.799365] Lustre: Skipped 1 previous similar message [ 4578.748095] Lustre: server umount lustre-MDT0000 complete [ 4581.857452] LustreError: 141963:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4581.880900] LustreError: 141963:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4582.676149] LustreError: 141943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837613 with bad export cookie 16209461885946357154 [ 4582.680833] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4582.688048] LustreError: 141943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4583.088834] Lustre: server umount lustre-MDT0001 complete [ 4594.338819] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4597.647768] Key type lgssc unregistered [ 4598.047544] LNet: 145902:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4598.077668] LNetError: 145902:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4598.109262] LNet: Removed LNI 192.168.202.155@tcp [ 4599.117486] Key type .llcrypt unregistered [ 4599.120387] Key type ._llcrypt unregistered [ 4622.322588] Key type ._llcrypt registered [ 4622.324139] Key type .llcrypt registered [ 4622.424222] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4623.910705] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4623.965642] alg: No test for adler32 (adler32-zlib) [ 4625.069793] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 4625.367815] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4627.072226] Key type lgssc registered [ 4628.287676] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4635.274868] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4635.293670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4636.771857] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4641.240832] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4645.862937] LustreError: 147313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4645.876683] LustreError: 147313:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4650.228828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4650.780465] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4655.504798] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4663.099521] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4669.000190] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4675.270571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4675.512612] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 4675.637363] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4680.700918] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4681.737082] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4682.242387] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4691.459819] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4697.553753] Lustre: *** cfs_fail_loc=159, val=0*** [ 4697.560597] Lustre: Skipped 1 previous similar message [ 4714.103706] Lustre: lustre-MDT0000: Client 31f871fc-07d7-4d5d-9f6d-07d74bb1b185 (at 192.168.202.55@tcp) reconnecting [ 4721.637848] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4721.645755] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4721.666235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4722.663100] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4722.687657] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4722.692997] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4722.699108] Lustre: Skipped 1 previous similar message [ 4726.727284] Lustre: server umount lustre-OST0000 complete [ 4732.899329] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4732.903667] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4732.923675] Lustre: Skipped 1 previous similar message [ 4737.297896] Lustre: server umount lustre-MDT0000 complete [ 4738.018930] LustreError: 147312:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4738.048133] LustreError: 147312:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4741.211618] LustreError: 147293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837772 with bad export cookie 6319065692647518917 [ 4741.217710] LustreError: 147293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4741.218371] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4741.484580] Lustre: server umount lustre-MDT0001 complete [ 4753.812368] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4756.391989] Key type lgssc unregistered [ 4756.666428] LNet: 150978:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4756.674877] LNetError: 150978:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4756.694202] LNet: Removed LNI 192.168.202.155@tcp [ 4757.385484] Key type .llcrypt unregistered [ 4757.388204] Key type ._llcrypt unregistered [ 4781.147541] Key type ._llcrypt registered [ 4781.149575] Key type .llcrypt registered [ 4781.300665] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4782.685619] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4782.710241] alg: No test for adler32 (adler32-zlib) [ 4783.795450] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 4784.168538] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4785.896227] Key type lgssc registered [ 4787.399429] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4794.752281] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4794.810972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4796.253928] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4800.909397] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4806.626955] LustreError: 152388:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4806.665796] LustreError: 152388:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4811.750754] LustreError: 152387:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4811.858472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4812.866133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4817.780396] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4827.504382] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4834.513621] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4841.913132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4842.555550] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4850.439466] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4856.885887] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4857.746318] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4866.129454] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4889.935556] Lustre: lustre-MDT0001: evicting ffe2d0b5-32ce-4fad-af72-e7fd43f309d5 (at 192.168.202.55@tcp) by administrative request [ 4889.943022] Lustre: Skipped 1 previous similar message [ 4922.339056] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4922.348407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4922.368324] Lustre: Skipped 1 previous similar message [ 4924.606180] Lustre: server umount lustre-OST0000 complete [ 4930.529799] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4930.538979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4930.562387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4930.569848] Lustre: Skipped 1 previous similar message [ 4932.580052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4932.598560] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4934.840693] Lustre: server umount lustre-MDT0000 complete [ 4937.699564] LustreError: 153647:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4937.715246] LustreError: 153647:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4939.097214] LustreError: 152369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837970 with bad export cookie 10575212181505096098 [ 4939.114320] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4939.477721] Lustre: server umount lustre-MDT0001 complete [ 4952.137846] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4955.360701] Key type lgssc unregistered [ 4955.639731] LNet: 156483:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4955.671582] LNetError: 156483:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4956.716138] LNet: Removed LNI 192.168.202.155@tcp [ 4957.652705] Key type .llcrypt unregistered [ 4957.655290] Key type ._llcrypt unregistered [ 5037.266776] Key type ._llcrypt registered [ 5037.271963] Key type .llcrypt registered [ 5037.427889] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5038.698433] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5038.752821] alg: No test for adler32 (adler32-zlib) [ 5040.081128] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 5040.348063] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 5042.064186] Key type lgssc registered [ 5043.546376] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5052.582531] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5052.614860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5054.135988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5059.519390] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5064.682028] LustreError: 158970:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5064.702368] LustreError: 158970:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5069.186624] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5069.450286] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5074.333765] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5081.719925] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5087.782978] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5094.591913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5094.991598] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5099.146230] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5101.704385] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5111.067035] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5114.563650] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5125.110844] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5125.115957] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5125.129788] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5125.131213] Lustre: Skipped 1 previous similar message [ 5130.210691] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5130.219322] Lustre: Skipped 2 previous similar messages [ 5130.783731] Lustre: server umount lustre-OST0000 complete [ 5135.339329] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5135.356834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5135.362718] Lustre: Skipped 1 previous similar message [ 5140.450774] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5140.461862] Lustre: Skipped 1 previous similar message [ 5145.573528] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5145.579470] Lustre: Skipped 1 previous similar message [ 5149.152993] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5149.447179] Lustre: server umount lustre-MDT0000 complete [ 5150.700439] LustreError: 159701:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5153.910625] LustreError: 158950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788838185 with bad export cookie 5627321860782682589 [ 5153.923714] LustreError: 158950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5153.929044] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5154.330973] Lustre: server umount lustre-MDT0001 complete [ 5168.545114] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 5171.578985] Key type lgssc unregistered [ 5171.891991] LNet: 162535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5171.909481] LNetError: 162535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5171.927978] LNet: Removed LNI 192.168.202.155@tcp [ 5173.004238] Key type .llcrypt unregistered [ 5173.006617] Key type ._llcrypt unregistered [ 5197.663757] Key type ._llcrypt registered [ 5197.665740] Key type .llcrypt registered [ 5197.793245] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5198.772576] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5198.869432] alg: No test for adler32 (adler32-zlib) [ 5199.926627] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 5200.175753] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 5201.840300] Key type lgssc registered [ 5203.425853] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5219.151447] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 23:30:48 (1788838248) [ 5236.481847] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 5249.693236] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5291.871811] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5291.895963] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5293.187898] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5293.213915] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5293.297855] Lustre: lustre-MDT0000: new disk, initializing [ 5293.375846] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5293.400269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5298.649725] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5310.527291] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5310.683531] Lustre: 166797:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5310.774242] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5310.777528] Lustre: Skipped 1 previous similar message [ 5310.854918] Lustre: lustre-MDT0001: new disk, initializing [ 5310.962638] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5311.025338] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5311.048666] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5316.179475] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5327.406582] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5335.255368] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5345.425119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5345.824987] Lustre: lustre-OST0000: new disk, initializing [ 5345.829346] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5345.844625] Lustre: 168768:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5345.991114] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5351.818227] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5355.708055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5355.726757] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5355.815270] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5362.400423] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5376.488247] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5376.496035] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5376.506209] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5378.854965] Lustre: server umount lustre-OST0000 complete [ 5386.744510] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5386.752567] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5386.772669] Lustre: Skipped 2 previous similar messages [ 5386.807760] Lustre: Skipped 2 previous similar messages [ 5389.582354] Lustre: server umount lustre-MDT0000 complete [ 5394.229565] LustreError: 166789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788838425 with bad export cookie 873975488568395918 [ 5394.237787] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5394.243366] LustreError: 166789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5394.754381] Lustre: server umount lustre-MDT0001 complete [ 5409.265939] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 5412.045648] Key type lgssc unregistered [ 5412.304724] LNet: 170672:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5412.311480] LNetError: 170672:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5412.334258] LNet: Removed LNI 192.168.202.155@tcp [ 5413.316739] Key type .llcrypt unregistered [ 5413.319537] Key type ._llcrypt unregistered [ 5443.790415] Key type ._llcrypt registered [ 5443.793494] Key type .llcrypt registered [ 5443.974994] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5445.819682] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5445.886283] alg: No test for adler32 (adler32-zlib) [ 5447.173906] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 5447.591969] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 5449.313529] Key type lgssc registered [ 5450.711985] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5460.930516] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5460.967586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5462.616165] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5467.491654] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5472.745611] LustreError: 172084:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5472.762048] LustreError: 172084:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5477.861765] LustreError: 172085:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5478.617960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5479.198595] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5485.310503] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5494.092407] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5500.642524] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5508.903164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5509.239130] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5516.971848] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5526.814322] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5532.796638] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5544.578059] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5544.633191] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5544.665420] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5544.682131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.707423] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5546.126874] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5546.146706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5546.170630] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5546.183338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.201238] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5547.735901] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5547.791690] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5547.809190] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5547.816932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.832895] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5548.827572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.542197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.555494] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5550.560399] Lustre: Skipped 2 previous similar messages [ 5550.577638] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5550.585887] Lustre: Skipped 2 previous similar messages [ 5550.589307] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5550.593148] Lustre: Skipped 2 previous similar messages [ 5550.596971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.605083] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5550.613442] Lustre: Skipped 2 previous similar messages [ 5551.380258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.524378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.619710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.834894] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5554.841637] Lustre: Skipped 3 previous similar messages [ 5554.860942] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5554.868753] Lustre: Skipped 3 previous similar messages [ 5554.872372] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5554.877521] Lustre: Skipped 3 previous similar messages [ 5554.881433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.892812] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5554.901102] Lustre: Skipped 3 previous similar messages [ 5556.102189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.997499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.049294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.135180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.244078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.413132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.492880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.828323] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5563.836249] Lustre: Skipped 7 previous similar messages [ 5563.846143] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5563.854205] Lustre: Skipped 7 previous similar messages [ 5563.857436] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5563.861285] Lustre: Skipped 7 previous similar messages [ 5563.864676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.869249] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5563.872111] Lustre: Skipped 7 previous similar messages [ 5564.879826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.094814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.162486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.951308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.375228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.592616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.665181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.689658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.841255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.707952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.698484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.628327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.616784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.735784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.697730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.704455] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5580.711843] Lustre: Skipped 15 previous similar messages [ 5580.723884] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5580.736891] Lustre: Skipped 15 previous similar messages [ 5580.740653] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5580.744810] Lustre: Skipped 15 previous similar messages [ 5580.748092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.758603] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5580.765374] Lustre: Skipped 15 previous similar messages [ 5581.910565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.123488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.060224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.088173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.078610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.938452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.779061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.684559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.564552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.696333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.767679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.156988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.634724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.301833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.542791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.564742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.868626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.851221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.023914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.096336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.813936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.613863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.680183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.056353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.290930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.433410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.718796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.948191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.315049] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5613.331637] Lustre: Skipped 28 previous similar messages [ 5613.369243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5613.381228] Lustre: Skipped 28 previous similar messages [ 5613.386622] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5613.403554] Lustre: Skipped 28 previous similar messages [ 5613.421991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.452646] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5613.467262] Lustre: Skipped 28 previous similar messages [ 5614.969748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.166653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.161840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.430286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.477828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.622260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.744382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.930917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.027924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.915604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.863733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.782385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.707529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.772647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.629538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.839322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.841687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.095901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.014572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.120550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.146362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.113351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.203569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.441136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.305551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.196603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.215040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.101964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.027701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.753873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.615072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.564276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.541530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.459451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.394253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.317800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.327141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.320073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.481325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.573793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.649194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.050735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.426198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.618757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.817612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.441795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.580800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.762482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.975888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.848977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.312892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.542090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.815235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.783253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.691744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.213421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.256136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.282256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.388993] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5677.396146] Lustre: Skipped 58 previous similar messages [ 5677.415265] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5677.424303] Lustre: Skipped 58 previous similar messages [ 5677.427949] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5677.432314] Lustre: Skipped 58 previous similar messages [ 5677.436100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.319290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.331537] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5678.341357] Lustre: Skipped 59 previous similar messages [ 5679.522209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.349915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.464853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.250589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.405146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.711449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.519036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.595531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.917825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.239940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.815721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.170214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.477210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.129200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.567641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.976283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.440109] LustreError: 172104:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5699.464072] LustreError: 172104:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5699.469343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.914371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.644925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.883121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.396459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.719543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.739516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.962101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.419693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.889682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.458591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.134061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.178118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.439880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.668485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.078408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.481814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.726058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.940098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.138063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.162274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.394456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.697471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.260797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.302658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.623348] LustreError: 172104:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5730.637622] LustreError: 172104:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5730.655642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.205758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.856873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.436730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.586845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.585783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.753901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.903654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.169074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.227770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.295052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.606171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.634408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.662706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.532912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.522774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.598729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.606537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.746622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.817431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.923854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.985787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.940971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.926866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.914406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.906875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.001500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.009519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.133958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.169721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.114425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.083849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.157823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.004282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.039409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.057521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.114867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.189334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.195269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.198813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.189974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.261382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.297955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.469606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.488813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.496114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.572282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.558238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.934701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.836636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.937436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.600771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.764204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.686240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.891854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.711760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.801489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.888844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.916537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.992219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.128573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.587065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.670873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.715051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.575258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.793668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.865642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.950078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.079443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.929727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.980104] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5805.985191] Lustre: Skipped 112 previous similar messages [ 5806.006702] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5806.014152] Lustre: Skipped 112 previous similar messages [ 5806.017807] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5806.022133] Lustre: Skipped 112 previous similar messages [ 5806.024790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.879376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.892348] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5806.897493] Lustre: Skipped 112 previous similar messages [ 5807.919650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.781457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.695129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.605996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.738484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.788986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.781372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.779159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.898222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.872054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.983361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.939842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.898073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.816882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.926711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.802534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.640042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.670395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.883413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.777681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.693766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.505735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.416986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.254197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.190215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.152792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.131599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.106715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.054201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.802610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.020909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.079311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.241315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.213706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.139441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.102944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.842137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.833308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.845843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.933245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.813289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.787856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.704246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.776392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.704641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.548865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.385427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.300687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.182886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.228810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.016297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.045321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.100919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.026192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.939306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.849065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.760312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.641628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.612851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.512486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.408919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.121358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.899491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.853839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.605986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.398317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.259650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.001958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.741401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.441579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.259182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.975449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.738072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.733605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.547960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.511750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.344164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.195400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.996086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.833586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.572819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.312138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.865353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.519772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.478625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.224308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.973209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.789659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.601488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.538931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.583801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.333330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.035675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.780695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.624290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.356963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.121170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.932699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.699632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.679329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.647851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.504089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.184610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.831570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.391178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.978684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.711907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.425475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.032244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.892463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.840754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.952500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.960777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.233703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.232476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.499642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.839657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.140741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.160954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.339776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.332197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.157891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.118323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.207896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.174470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.144497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.171192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.017037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.961540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.117451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.092107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.248774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.267533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.325757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.564915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.485482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.479037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.713917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.766947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.732899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.816205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.668853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.950932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.223844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.267229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.054979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.969519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.965302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.876904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.886605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.905862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.874795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.850942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.914294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.831879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.938359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.952843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.932461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.731395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.839652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.820903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.933614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.816847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.720544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.773339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.856436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.726176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.786391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.807215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.019736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.968590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.108563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.330058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.398423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.406151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.228366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.140979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.967550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.912859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.838089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.752055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.612336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.307660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.013078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.901309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.719909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.490060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.253235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.092225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.922294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.872838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.702629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.784634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.613340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.527829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.466097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.358541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.240963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.041328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.778749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.520691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.356424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.123940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.922510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.849137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.755404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.634458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.362392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.223394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.171066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.069668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.825930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.564510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.537827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.544944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.414884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.218281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.176402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.084772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.045882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.051584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.912907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.967444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.005815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.092644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.046144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.862700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.901816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.981627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.795301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.683240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.439333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.275389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.065129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.827178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.585684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.377708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.169460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.109086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.958135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.954499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.193343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.186577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.137208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.886784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.953324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.887817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.752078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.684286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.656191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.581117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.697990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.706205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.770675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.726507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.838810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.973977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.956269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.969981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.149798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.227892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.204394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.295322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.221981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.378873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.380642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.279826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.206723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.223639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.101769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.926892] LustreError: 172104:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6058.933152] LustreError: 172104:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6058.942173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.656708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.644365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.349508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.309503] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6062.313162] Lustre: Skipped 275 previous similar messages [ 6062.326606] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6062.336970] Lustre: Skipped 275 previous similar messages [ 6062.340768] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6062.344894] Lustre: Skipped 275 previous similar messages [ 6062.347844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.096355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.108218] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6063.121212] Lustre: Skipped 275 previous similar messages [ 6064.119608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.956919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.901541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.752930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.736485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.757352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.664658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.283256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.993694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.667680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.451270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.140461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.878467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.644092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.432638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.380491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.365833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.287928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.200130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.241994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.199055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.179404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.068349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.133296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.997647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.893804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.753095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.630292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.456267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.342755] LustreError: 172104:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6089.353073] LustreError: 172104:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6089.366736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.349615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.344247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.203192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.944704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.758566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.581518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.328196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.421551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.325082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.428382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.299858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.193521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.077263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.051355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.854520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.668863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.459616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.425126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.373451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.223925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.126411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.317136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.409535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.521178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.235398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.254835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.245417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.301806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.153209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.015090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.825477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.852634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.739956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.791535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.747752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.848962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.863220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.935420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.803806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.741502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.030979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.162644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.131955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.189606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.218934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.215333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.332104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.273810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.390844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.329633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.425227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.321740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.295983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.234510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.192907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.274583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.138530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.183171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.147776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.245916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.053214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.296867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.650112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.851721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.827256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.096116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.183578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.278053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.455901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.548312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.854520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.299811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.579719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.704418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.924260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.238821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.733853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.113823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.202116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.422402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.068429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.490367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.813567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.336438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.121701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.057439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.207578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.463826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.670807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.740406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.060811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.664623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.281888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.576432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.543492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.726612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.442533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.103668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.924452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.050089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.191066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.479880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.517945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.875532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.843905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.008488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.047512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.199363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.527866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.675353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.720365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.650719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.124472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.524656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.650829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.935045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.731425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.334333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.883228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.255736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.507249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.015339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.203161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.392401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.516896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.736130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.859541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.897875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.073659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.213504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.456227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.703445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.923255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.190923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.391650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.720295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.958905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.165509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.350737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.653177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.056316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.645947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.298824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.859684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.895347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.032772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.326819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.600442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.759266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.486402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.127270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.919121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.297226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.565718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.871910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.272647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.259148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.583643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.816533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.237898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.536753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.533528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.009384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.472552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.699464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.026881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.951720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.133808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.355845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.724289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.340389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.903256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.223473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.525979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.520711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.770652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.374326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.629964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.617299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.942140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.847935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.983722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.176389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.395900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.363614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.538494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.483450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.641665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.674966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.048490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.958374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.935296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.215145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.567315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.677570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.004600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.385829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.011891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.311919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.439769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.026620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.069537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.763950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.737714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.012504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.791610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.953418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.421718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.255321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.681460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.371715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.547675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.617143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.882934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.899569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.035613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.174642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.216722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.564983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.787789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.870696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.052629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.400625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.500377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.693258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.030659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.934286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.256450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.214803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.464310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.744957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.769329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.870318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.947177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.053073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.289633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.426170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.482623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.699933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.804656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.145885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.125488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.350741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.329420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.761547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.971252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.943697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.122734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.208028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.406249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.593577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.825450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.894521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.046135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.105294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.469980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.463540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.519915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.350776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.318929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.439933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.417763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.722895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.776260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.914215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.884581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.044355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.326468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.494911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.524657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.604975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.713343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.668274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.800404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.657456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.751888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.889618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.844342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.985394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.899698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.872699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.867041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.981954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.234713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.184150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.336658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.329651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.334251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.339802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.227691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.350560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.514142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.756143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.811380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.849416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.919774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.095768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.152637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.160592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.308340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.116605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.122198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.976284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.011529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.945686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.097605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.938358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.964889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.798435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.866904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.009052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.989727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.888776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.957905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.931228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.902649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.878045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.981089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.201858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.014930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.008596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.869956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.860361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.784359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.942818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.866772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.150667] LustreError: 172104:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6463.164725] LustreError: 172104:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6463.177089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.313582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.574659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.587914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.745350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.931187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.095632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.143253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.367441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.485257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.478708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.605246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.492241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.604608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.656969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.618964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.551711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.580349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.566968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.609955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.706199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.880289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.815729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.901556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.005943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.932443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.052320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.170626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.334145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.526684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.800497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.055326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.287555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.491625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.003597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.362711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.493807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.571504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.807949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.323784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.468595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.771915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.875143] LustreError: 172104:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6510.887684] LustreError: 172104:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6510.897272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.868696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.016477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.225853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.228848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.117713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.127310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.203855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.266782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.392789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.248663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.078702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.908160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.857983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.832795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.922549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.153560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.100839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.937066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.987337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.005443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.747870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.693732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.825890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.842852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.907580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.742424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.555631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.406456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.487486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.488711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.538430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.317601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.267929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.471263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.614423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.838694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.156501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.220465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.328732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.727741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.420083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.701285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.899680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.375629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.800305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.896038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.890031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.969724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.291832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.411188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.573574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.882915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.632606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.758431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.826593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.664854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.855239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.751115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.853416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.044637] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6575.047254] Lustre: Skipped 459 previous similar messages [ 6575.056269] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6575.064181] Lustre: Skipped 459 previous similar messages [ 6575.071066] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6575.075132] Lustre: Skipped 459 previous similar messages [ 6575.082499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.894963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.934054] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6575.936930] Lustre: Skipped 459 previous similar messages [ 6576.826131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.893868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.909733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.759851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.751076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.686691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.574641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.600870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.551738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.536322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.588946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.593161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.547911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.355504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.401150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.325051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.287380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.271776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.245107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.201985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.086651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.970498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.792729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.751982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.692120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.455677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.365661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.258966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7005.243988] LustreError: 173911:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7005.263417] LustreError: 172104:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7622.114778] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7622.118250] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7622.124473] Lustre: Skipped 30 previous similar messages [ 7622.131493] Lustre: Skipped 1 previous similar message [ 7627.236569] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7627.242924] Lustre: Skipped 1 previous similar message [ 7627.972441] Lustre: server umount lustre-OST0000 complete [ 7634.918483] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7634.937227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7637.478578] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7639.390064] Lustre: server umount lustre-MDT0000 complete [ 7642.594836] LustreError: 172084:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7642.607160] LustreError: 172084:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7643.449933] LustreError: 172065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788840674 with bad export cookie 12019609129495145851 [ 7643.474520] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7643.479044] LustreError: 172065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7643.997221] Lustre: server umount lustre-MDT0001 complete [ 7656.539545] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 7659.527905] Key type lgssc unregistered [ 7659.802581] LNet: 227396:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7659.814369] LNetError: 227396:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7660.851868] LNet: Removed LNI 192.168.202.155@tcp [ 7661.705704] Key type .llcrypt unregistered [ 7661.708127] Key type ._llcrypt unregistered [ 7688.671713] Key type ._llcrypt registered [ 7688.676797] Key type .llcrypt registered [ 7688.876589] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7690.856784] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7690.949411] alg: No test for adler32 (adler32-zlib) [ 7692.081370] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 7692.424839] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 7694.185842] Key type lgssc registered [ 7695.666379] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7705.092497] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7705.134274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7707.076704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7712.810581] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7717.348406] LustreError: 228811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7717.364498] LustreError: 228811:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7722.098113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7722.438804] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7726.486915] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7734.938276] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7740.107605] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7747.713508] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7748.145754] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7752.249543] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7755.476328] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7764.729594] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7773.751608] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7776.017358] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7776.033597] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7776.039294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7776.056971] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7776.062619] LustreError: 228831:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7782.883246] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7782.897425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7782.898338] Lustre: Skipped 1 previous similar message [ 7782.909829] Lustre: Skipped 1 previous similar message [ 7784.414733] Lustre: server umount lustre-OST0000 complete [ 7796.152183] LustreError: 228831:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7819.165272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7819.649793] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7823.741688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7823.760530] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7823.763790] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7823.779977] Lustre: Skipped 1 previous similar message [ 7826.751266] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7835.504920] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7836.886892] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7842.832623] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9335d0e35800.ost_server_uuid 50 [ 7844.685653] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9335d0e35800.ost_server_uuid in FULL state after 0 sec [ 7849.455379] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7849.457821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7849.468180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7849.486858] Lustre: Skipped 1 previous similar message [ 7849.497856] Lustre: Skipped 2 previous similar messages [ 7854.562854] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7854.575192] Lustre: Skipped 1 previous similar message [ 7855.196167] Lustre: server umount lustre-OST0000 complete [ 7859.685331] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7859.694204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7859.702216] Lustre: Skipped 1 previous similar message [ 7859.718788] Lustre: Skipped 1 previous similar message [ 7864.801162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7864.814119] Lustre: Skipped 1 previous similar message [ 7865.718924] Lustre: server umount lustre-MDT0000 complete [ 7869.926230] LustreError: 228807:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7869.944924] LustreError: 228807:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7870.610222] LustreError: 228791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788840901 with bad export cookie 3601520009114426667 [ 7870.615799] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7871.059087] Lustre: server umount lustre-MDT0001 complete [ 7883.518741] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 7886.939623] Key type lgssc unregistered [ 7887.353507] LNet: 234018:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7887.367814] LNetError: 234018:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7888.423273] LNet: Removed LNI 192.168.202.155@tcp [ 7889.609246] Key type .llcrypt unregistered [ 7889.612228] Key type ._llcrypt unregistered [ 7925.317889] Key type ._llcrypt registered [ 7925.319189] Key type .llcrypt registered [ 7925.544711] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7927.514571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7927.587282] alg: No test for adler32 (adler32-zlib) [ 7928.772618] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 7929.179618] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 7930.936200] Key type lgssc registered [ 7932.556516] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7942.787888] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7942.811124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7944.425442] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7949.654172] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7953.902594] LustreError: 235745:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7953.916585] LustreError: 235745:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7958.866147] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7959.009940] LustreError: 235746:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7959.252385] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7963.751150] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7966.742099] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7973.890810] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7974.365410] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7981.206906] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7990.766633] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7991.042907] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7991.213502] Lustre: lustre-OST0001: new disk, initializing [ 7991.218460] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7991.223288] Lustre: 238053:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7991.334560] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7998.444409] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8000.099706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8000.110920] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8000.231645] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8010.393505] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8018.333057] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8036.324955] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8036.330339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8036.334782] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8036.349976] Lustre: Skipped 3 previous similar messages [ 8036.353185] Lustre: Skipped 2 previous similar messages [ 8041.132092] Lustre: server umount lustre-MDT0000 complete [ 8041.447457] LustreError: 236777:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8041.461377] LustreError: 236777:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8045.424196] LustreError: 235726:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841076 with bad export cookie 7673683909178213847 [ 8045.439722] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8045.441375] LustreError: 235726:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8046.052906] Lustre: server umount lustre-MDT0001 complete [ 8061.520540] Lustre: server umount lustre-OST0000 complete [ 8062.432804] Lustre: 235007:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788841077/real 1788841077] req@ffff92d399d88e00 x1875735698566400/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788841093 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8062.460198] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8065.921839] Lustre: server umount lustre-OST0001 complete [ 8089.543263] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8100.502993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8100.929073] LustreError: 241163:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8100.953374] LustreError: 241163:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8101.042747] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8105.293776] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8106.474409] LustreError: 241164:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8114.333302] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8114.798891] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8120.461755] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8123.827310] Lustre: 242305:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8130.844230] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8131.246335] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8137.430698] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8139.444399] LustreError: 242658:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8139.476296] LustreError: 242658:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8145.446851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8146.711168] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8152.689300] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8160.449366] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8164.039530] Lustre: 244176:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8176.098200] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8176.104788] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8176.116544] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8176.121213] Lustre: Skipped 1 previous similar message [ 8177.642355] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8177.657744] Lustre: Skipped 3 previous similar messages [ 8180.835704] Lustre: server umount mylustre-MDT0000 complete [ 8182.757152] LustreError: 242381:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8182.779264] LustreError: 242381:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 8184.961388] LustreError: 241139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841216 with bad export cookie 7673683909178216619 [ 8184.967954] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8184.983851] LustreError: 241139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8185.345627] Lustre: server umount mylustre-MDT0001 complete [ 8200.536769] Lustre: server umount mylustre-OST0000 complete [ 8203.760341] Lustre: 235007:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788841219/real 1788841219] req@ffff92d4899b2300 x1875735698627200/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788841235 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8203.827204] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8203.847082] Lustre: Skipped 3 previous similar messages [ 8205.554928] Lustre: server umount mylustre-OST0001 complete [ 8229.832914] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8240.072882] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8240.511167] LustreError: 246485:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8240.577231] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8240.582835] Lustre: Skipped 1 previous similar message [ 8245.902802] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8256.062915] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8262.134909] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8265.098961] Lustre: 247625:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8272.961417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8280.160978] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8285.550600] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8289.074894] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8296.326161] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8304.137992] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8307.578920] Lustre: 249499:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8315.372284] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8315.385966] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8315.393349] Lustre: Skipped 3 previous similar messages [ 8316.522770] Lustre: server umount tfs-MDT0000 complete [ 8320.484422] LustreError: 249266:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8320.500830] LustreError: 249266:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8320.556993] LustreError: 246462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841351 with bad export cookie 7673683909178218285 [ 8320.573234] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8320.579305] LustreError: 246462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8321.096938] Lustre: server umount tfs-MDT0001 complete [ 8334.938869] Lustre: server umount tfs-OST0000 complete [ 8349.548677] Lustre: server umount tfs-OST0001 complete [ 8376.812364] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8388.602741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8389.039193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8389.046747] Lustre: Skipped 3 previous similar messages [ 8393.667700] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8402.358140] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8407.338462] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8410.396403] Lustre: 252891:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8418.112640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8426.234031] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8433.934511] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 8437.020145] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8444.953981] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8453.708498] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8457.612673] Lustre: 254761:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8465.375269] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 00:24:54 (1788841494) [ 8483.308714] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8483.316030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8483.348421] Lustre: Skipped 5 previous similar messages [ 8483.354545] Lustre: Skipped 2 previous similar messages [ 8487.250042] Lustre: server umount lustre-MDT0000 complete [ 8488.423827] LustreError: 251750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8488.444814] LustreError: 251750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8492.784077] LustreError: 254763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841524 with bad export cookie 7673683909178219720 [ 8492.787100] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8492.802547] LustreError: 254763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8493.544203] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8493.559079] Lustre: Skipped 2 previous similar messages [ 8507.361406] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8507.632802] Lustre: server umount lustre-MDT0001 complete [ 8527.328282] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8527.600436] Lustre: server umount lustre-OST0000 complete [ 8546.273537] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8546.505777] Lustre: server umount lustre-OST0001 complete [ 8551.987690] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 8565.588932] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8604.476373] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8604.766808] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8604.795138] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8604.897419] Lustre: lustre-MDT0000: new disk, initializing [ 8605.003491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8605.006552] Lustre: Skipped 3 previous similar messages [ 8605.028962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8609.685268] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8620.906633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8621.040468] Lustre: 257954:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8621.104867] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8621.108423] Lustre: Skipped 1 previous similar message [ 8621.175542] Lustre: lustre-MDT0001: new disk, initializing [ 8621.304207] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8621.311622] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8625.698715] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8636.834495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8637.072430] Lustre: lustre-OST0000: new disk, initializing [ 8637.075945] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8637.100040] Lustre: 259538:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8638.531579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8638.565690] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8638.673346] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8644.807697] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8658.342254] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8658.632448] Lustre: lustre-OST0001: new disk, initializing [ 8658.636631] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8658.643871] Lustre: 260409:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8660.079942] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8660.108296] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8660.243278] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8668.785663] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8686.502975] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 00:28:34 (1788841714) [ 8703.115803] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 00:28:52 (1788841732) [ 8720.866138] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8720.869026] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8720.881760] LustreError: Skipped 1 previous similar message [ 8720.884658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8720.895313] Lustre: Skipped 4 previous similar messages [ 8720.897242] Lustre: Skipped 6 previous similar messages [ 8724.683623] Lustre: server umount lustre-OST0000 complete [ 8729.059806] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8734.986689] Lustre: server umount lustre-MDT0000 complete [ 8739.112103] LustreError: 257946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841770 with bad export cookie 7673683909178221057 [ 8739.122398] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8739.127382] LustreError: 257946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8745.758454] Lustre: server umount lustre-MDT0001 complete [ 8758.396939] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8834.157224] Lustre: server umount lustre-OST0001 complete [ 8841.867492] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 8861.124370] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8905.272899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8905.525337] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8905.559973] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8905.664311] Lustre: lustre-MDT0000: new disk, initializing [ 8905.723490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8905.734795] Lustre: Skipped 3 previous similar messages [ 8905.765060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8910.222715] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8921.120141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8921.347668] Lustre: 265266:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8921.481495] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8921.487055] Lustre: Skipped 1 previous similar message [ 8921.689880] Lustre: lustre-MDT0001: new disk, initializing [ 8922.049075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8922.066044] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8928.309376] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8939.983419] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8946.868983] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8953.480372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8953.864690] Lustre: lustre-OST0000: new disk, initializing [ 8953.871975] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8953.878364] Lustre: 267237:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8955.030844] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8955.044608] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8955.228751] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8960.260945] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8970.127964] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8980.967940] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8980.974111] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8980.993541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8981.015653] Lustre: Skipped 5 previous similar messages [ 8981.031681] Lustre: Skipped 9 previous similar messages [ 8984.598941] Lustre: server umount lustre-OST0000 complete [ 8995.776444] Lustre: server umount lustre-MDT0000 complete [ 8996.322923] LustreError: 265275:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8996.347700] LustreError: 265275:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 18 previous similar messages [ 8999.553666] LustreError: 265261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842030 with bad export cookie 7673683909178222527 [ 8999.563688] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8999.835913] Lustre: server umount lustre-MDT0001 complete [ 9011.590908] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9014.895826] Key type lgssc unregistered [ 9015.324710] LNet: 268997:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9015.338533] LNetError: 268997:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9015.351744] LNet: Removed LNI 192.168.202.155@tcp [ 9016.335536] Key type .llcrypt unregistered [ 9016.338456] Key type ._llcrypt unregistered [ 9048.389269] Key type ._llcrypt registered [ 9048.393250] Key type .llcrypt registered [ 9048.518679] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9049.875503] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9049.960042] alg: No test for adler32 (adler32-zlib) [ 9051.041380] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 9051.221631] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9052.896964] Key type lgssc registered [ 9054.434677] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9063.576941] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9063.594483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9065.125105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9069.960668] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9075.696916] LustreError: 270459:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9075.728488] LustreError: 270459:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9079.901622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9080.395794] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9085.054894] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9093.871943] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9100.556489] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9108.820748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9109.270409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9110.348361] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9117.660536] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9127.073697] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9128.391834] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9136.097403] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9136.106228] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9136.115122] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9136.126468] Lustre: Skipped 1 previous similar message [ 9136.148343] Lustre: Skipped 1 previous similar message [ 9141.226549] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9141.242175] Lustre: Skipped 1 previous similar message [ 9145.312158] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9145.597072] Lustre: server umount lustre-OST0000 complete [ 9151.466342] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9151.481732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9151.493479] Lustre: Skipped 1 previous similar message [ 9156.390483] LustreError: 273296:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x70616d65646f6e:0x4:0x0].0x0 (00000000683ac17e) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 9156.579693] LustreError: 270460:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9156.588971] LustreError: 270460:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9156.622260] Lustre: server umount lustre-MDT0000 complete [ 9161.288741] LustreError: 270441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842192 with bad export cookie 462328424506150945 [ 9161.300784] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9161.306237] LustreError: 270441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9162.042923] Lustre: server umount lustre-MDT0001 complete [ 9174.345392] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9177.021245] Key type lgssc unregistered [ 9177.316041] LNet: 274046:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9177.323613] LNetError: 274046:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9177.345311] LNet: Removed LNI 192.168.202.155@tcp [ 9178.356841] Key type .llcrypt unregistered [ 9178.359120] Key type ._llcrypt unregistered [ 9198.090801] Key type ._llcrypt registered [ 9198.097477] Key type .llcrypt registered [ 9198.277027] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9199.825067] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9200.080950] alg: No test for adler32 (adler32-zlib) [ 9201.363111] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 9201.618857] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9203.290349] Key type lgssc registered [ 9204.442823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9213.802480] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9213.834269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9229.408397] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9234.530211] LustreError: 275237:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.155@tcp: failed processing log, type 4: rc = -110 [ 9261.217664] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9269.170336] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9277.376314] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9279.912486] Lustre: server umount lustre-OST0000 complete [ 9300.570333] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9304.104905] Key type lgssc unregistered [ 9304.462643] LNet: 276869:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9304.469989] LNetError: 276869:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9304.493249] LNet: Removed LNI 192.168.202.155@tcp [ 9305.554251] Key type .llcrypt unregistered [ 9305.562276] Key type ._llcrypt unregistered [ 9354.914728] Key type ._llcrypt registered [ 9354.916733] Key type .llcrypt registered [ 9355.048286] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9356.330327] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9356.394893] alg: No test for adler32 (adler32-zlib) [ 9357.494988] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 9357.840638] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9359.576194] Key type lgssc registered [ 9361.203855] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9375.874981] loop0: detected capacity change from 0 to 204800000 [ 9376.105932] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9376.120200] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9376.403230] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9377.633265] loop0: detected capacity change from 0 to 204800000 [ 9377.800887] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9377.852721] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9378.215540] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9379.214898] loop0: detected capacity change from 0 to 204800000 [ 9379.417463] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9379.438212] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9379.569390] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9381.064449] loop0: detected capacity change from 0 to 204800000 [ 9381.264972] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9381.282667] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9381.365390] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9382.472450] loop0: detected capacity change from 0 to 204800000 [ 9382.504864] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9387.969827] loop0: detected capacity change from 0 to 204800000 [ 9388.046495] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9392.627531] loop0: detected capacity change from 0 to 204800000 [ 9392.667129] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9396.870121] loop0: detected capacity change from 0 to 204800000 [ 9396.915252] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9401.382132] loop0: detected capacity change from 0 to 204800000 [ 9401.466629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9401.506403] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9401.570786] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9401.577309] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9401.585177] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9401.592723] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9401.599985] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9401.606959] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9401.613743] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9401.620298] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9401.626841] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9401.633939] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9401.641669] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9401.648763] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9401.655759] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9401.664950] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9407.669613] Lustre: server umount MGS complete [ 9408.562333] loop0: detected capacity change from 0 to 204800000 [ 9408.640131] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9409.115842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9409.151061] Lustre: lustre-MDT0000: Aborting client recovery [ 9409.178307] LustreError: 280659:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 9409.875361] loop1: detected capacity change from 0 to 204800000 [ 9410.018106] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9410.242127] Lustre: 280716:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9410.254858] Lustre: 280716:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9410.262458] Lustre: 280716:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9410.271727] Lustre: 280716:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9410.283446] Lustre: 280716:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9410.295288] Lustre: 280716:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9410.305650] Lustre: 280716:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9410.330700] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9410.553607] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9410.601099] Lustre: lustre-MDT0001: Aborting client recovery [ 9411.619495] loop2: detected capacity change from 0 to 204800000 [ 9411.799837] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9411.869104] Lustre: 280795:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9411.878518] Lustre: 280795:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9411.888633] Lustre: 280795:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9411.908592] Lustre: 280795:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9411.930309] Lustre: 280795:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9411.937351] Lustre: 280795:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9411.948934] Lustre: 280795:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9412.103671] Lustre: Found index 0 for lustre-OST0000, updating log [ 9412.198432] Lustre: 280813:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9412.284819] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9412.308919] Lustre: lustre-OST0000: Aborting recovery [ 9413.265330] loop3: detected capacity change from 0 to 204800000 [ 9413.418491] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9413.473165] Lustre: 280878:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9413.493883] Lustre: 280878:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9413.505357] Lustre: 280878:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9413.519290] Lustre: 280878:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9413.540201] Lustre: 280878:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9413.550144] Lustre: 280878:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9413.560437] Lustre: 280878:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9413.584810] Lustre: Found index 1 for lustre-OST0001, updating log [ 9413.645431] Lustre: 280883:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9413.720348] Lustre: lustre-OST0001: Aborting recovery [ 9418.785724] LustreError: 280660:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9419.753358] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9419.753496] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9419.845320] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9419.854832] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9420.788199] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9420.788695] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9421.523757] LustreError: 280627:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 918us: rc = -2 [ 9422.114294] LustreError: 280628:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 5510us: rc = -2 [ 9422.133953] LustreError: 280628:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 21 previous similar messages [ 9436.129389] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9436.131377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9436.144339] Lustre: Skipped 2 previous similar messages [ 9436.158779] Lustre: Skipped 3 previous similar messages [ 9441.250297] LustreError: 280739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9441.267037] LustreError: 280739:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 9442.477897] Lustre: server umount lustre-MDT0000 complete [ 9443.509937] LustreError: 280611:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842474 with bad export cookie 13368396765323594737 [ 9443.513054] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9443.516777] LustreError: 280611:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9446.377356] LustreError: 280626:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9446.381027] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9446.383815] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9446.399330] LustreError: 280626:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9446.435265] Lustre: Skipped 2 previous similar messages [ 9448.417109] LustreError: 280625:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9448.417155] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9448.430865] LustreError: 280625:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9448.447820] Lustre: Skipped 2 previous similar messages [ 9450.138554] Lustre: server umount lustre-MDT0001 complete [ 9465.312251] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9465.573740] Lustre: server umount lustre-OST0000 complete [ 9473.063192] Lustre: server umount lustre-OST0001 complete [ 9481.154938] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 00:41:50 (1788842510) [ 9511.452749] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 9522.130357] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9557.788922] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9558.153202] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9558.193910] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9558.349277] Lustre: lustre-MDT0000: new disk, initializing [ 9558.426228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9558.433848] Lustre: Skipped 1 previous similar message [ 9558.446312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9562.924853] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9573.113301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9573.304809] Lustre: 284952:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9573.387092] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9573.402410] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 9573.403775] Lustre: Skipped 1 previous similar message [ 9573.518554] Lustre: lustre-MDT0001: new disk, initializing [ 9573.628939] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9573.687277] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9573.706820] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9578.791982] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9589.159256] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9595.871916] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9603.924937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9604.269696] Lustre: lustre-OST0000: new disk, initializing [ 9604.272919] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9604.278447] Lustre: 286923:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9604.391813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9610.746499] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9611.858260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9611.867628] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9612.092316] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9620.471634] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9626.169310] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9637.868063] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9644.566581] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9654.793332] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9694.177884] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9694.179068] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9694.189584] LustreError: Skipped 1 previous similar message [ 9694.191600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9694.216518] Lustre: Skipped 1 previous similar message [ 9699.306508] LustreError: 286909:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9699.323367] LustreError: 286909:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9699.404668] Lustre: server umount lustre-OST0000 complete [ 9704.419023] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9704.428395] Lustre: Skipped 1 previous similar message [ 9704.433210] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9704.436461] Lustre: Skipped 2 previous similar messages [ 9710.083631] Lustre: server umount lustre-MDT0000 complete [ 9714.658636] LustreError: 284966:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9714.684366] LustreError: 284966:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9715.247150] LustreError: 284946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842746 with bad export cookie 13368396765323597446 [ 9715.277453] LustreError: 284946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9715.304030] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9715.809262] Lustre: server umount lustre-MDT0001 complete [ 9726.112389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9731.699968] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9736.880091] Lustre: server umount MGS complete [ 9749.160044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9749.906974] LustreError: 290797:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9749.961750] LustreError: 290810:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9750.057138] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9754.602275] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9765.555691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9770.833159] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9782.240580] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9788.884198] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9798.522551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9799.157816] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9799.163433] Lustre: Skipped 1 previous similar message [ 9806.421432] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9815.961678] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9851.369634] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9851.375511] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9851.381034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9851.388121] Lustre: Skipped 1 previous similar message [ 9851.401794] Lustre: Skipped 3 previous similar messages [ 9853.695858] Lustre: server umount lustre-OST0000 complete [ 9858.019611] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9861.606461] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9861.616826] Lustre: Skipped 1 previous similar message [ 9872.354244] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9872.560071] Lustre: server umount lustre-MDT0000 complete [ 9875.949768] LustreError: 290793:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9875.988975] LustreError: 290793:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 9877.046875] LustreError: 293527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842908 with bad export cookie 13368396765323599392 [ 9877.054400] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9877.060148] LustreError: 293527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9877.542365] Lustre: server umount lustre-MDT0001 complete [ 9890.408617] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9893.762303] Key type lgssc unregistered [ 9894.056996] LNet: 294745:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9894.066668] LNetError: 294745:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9894.092783] LNet: Removed LNI 192.168.202.155@tcp [ 9895.174389] Key type .llcrypt unregistered [ 9895.178057] Key type ._llcrypt unregistered [ 9934.346578] Key type ._llcrypt registered [ 9934.351250] Key type .llcrypt registered [ 9934.553626] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 9950.603271] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9952.191353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9952.225204] alg: No test for adler32 (adler32-zlib) [ 9953.420628] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [ 9953.711975] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9955.392201] Key type lgssc registered [ 9956.642357] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9998.984533] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9999.023733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10000.360520] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10000.393593] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10000.472359] Lustre: lustre-MDT0000: new disk, initializing [10000.561739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10000.588174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10005.946779] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10017.405585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10017.541582] Lustre: 298705:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10017.592044] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10017.595363] Lustre: Skipped 1 previous similar message [10017.714476] Lustre: lustre-MDT0001: new disk, initializing [10017.820556] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10017.863555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10017.870640] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10023.122511] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10033.182771] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10039.276489] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10046.306098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10046.638377] Lustre: lustre-OST0000: new disk, initializing [10046.643035] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10046.650312] Lustre: 300672:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10046.737751] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10052.681256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10052.696177] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10052.773411] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10053.190283] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10064.032444] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10072.248787] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10084.638839] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10091.497209] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10098.507571] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10145.251059] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10145.263801] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10145.289282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10145.293852] Lustre: Skipped 1 previous similar message [10145.315896] Lustre: Skipped 1 previous similar message [10148.437158] Lustre: server umount lustre-OST0000 complete [10155.490248] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10155.501508] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10155.505111] Lustre: Skipped 1 previous similar message [10155.526959] Lustre: Skipped 1 previous similar message [10159.882427] Lustre: server umount lustre-MDT0000 complete [10165.247023] LustreError: 302338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843196 with bad export cookie 8249446852397146990 [10165.252617] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10165.257373] LustreError: 302338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10166.385916] Lustre: server umount lustre-MDT0001 complete [10178.789154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10184.496443] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10190.293529] Lustre: server umount MGS complete [10203.215858] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10203.849792] LustreError: 304541:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10203.887222] LustreError: 304554:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10203.880937] LustreError: 304541:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10204.086485] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10208.353439] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10209.249543] LustreError: 304542:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10209.255668] LustreError: 304542:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10214.378639] LustreError: 304541:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10219.064954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10219.692032] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10225.710697] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10234.467755] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10241.359353] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10249.805386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10250.408231] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10258.306945] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10267.961077] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10310.117911] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10310.128914] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10310.130500] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10310.138249] LustreError: Skipped 1 previous similar message [10310.157933] Lustre: Skipped 1 previous similar message [10315.261650] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10315.269659] Lustre: Skipped 2 previous similar messages [10316.064177] Lustre: server umount lustre-OST0000 complete [10321.893078] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10321.903077] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10321.926050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10327.945317] Lustre: server umount lustre-MDT0000 complete [10330.599726] LustreError: 305976:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10330.627383] LustreError: 305976:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10334.032361] LustreError: 304522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843365 with bad export cookie 8249446852397148873 [10334.057917] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10334.066447] LustreError: 304522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10334.967386] Lustre: server umount lustre-MDT0001 complete [10348.443872] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [10352.092989] Key type lgssc unregistered [10352.511904] LNet: 308509:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10352.518042] LNetError: 308509:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10353.580906] LNet: Removed LNI 192.168.202.155@tcp [10354.946817] Key type .llcrypt unregistered [10354.948940] Key type ._llcrypt unregistered [10380.547255] Key type ._llcrypt registered [10380.549761] Key type .llcrypt registered [10380.758101] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [10398.760234] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10399.443580] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10399.591982] alg: No test for adler32 (adler32-zlib) [10400.694566] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [10400.963611] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [10402.648774] Key type lgssc registered [10403.352824] Lustre: Echo OBD driver; http://www.lustre.org/ [10433.664562] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10450.069631] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10450.121917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10451.464595] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10451.506581] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10451.593993] Lustre: lustre-MDT0000: new disk, initializing [10451.679912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10451.698130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10458.268606] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10471.360766] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10471.607772] Lustre: lustre-OST0000: new disk, initializing [10471.610611] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10471.614250] Lustre: Skipped 1 previous similar message [10471.617982] Lustre: 312559:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10471.687417] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10479.614503] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10481.292766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10481.301504] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10481.330777] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11396.066618] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11396.076830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11401.184877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11404.447254] Lustre: server umount lustre-MDT0000 complete [11408.150834] LustreError: 311745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844439 with bad export cookie 13063368929358517854 [11408.163121] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11408.165605] LustreError: 311745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11408.317117] Lustre: server umount lustre-OST0000 complete [11435.530155] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [11445.429379] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [11488.605420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11488.899535] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11488.930824] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11489.049195] Lustre: lustre-MDT0000: new disk, initializing [11489.132135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11489.156251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11494.385967] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11505.282412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11505.463552] Lustre: 316926:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11505.565914] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11505.569592] Lustre: Skipped 1 previous similar message [11505.656350] Lustre: lustre-MDT0001: new disk, initializing [11505.737232] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11505.755628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11505.762080] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11510.422328] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11520.825540] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11525.745559] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11532.488141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11532.992833] Lustre: lustre-OST0000: new disk, initializing [11533.014987] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11533.046250] Lustre: 318893:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11533.240358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11540.092742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11540.100589] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11540.277361] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11540.910185] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11551.851275] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11557.696933] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11557.986397] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11563.787206] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11564.114964] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11571.177842] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11571.177958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11571.192914] LustreError: Skipped 1 previous similar message [11571.206779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11576.294784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11576.302477] Lustre: Skipped 2 previous similar messages [11580.384339] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11580.700719] Lustre: server umount lustre-OST0000 complete [11586.533388] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11586.540495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11586.542076] Lustre: Skipped 2 previous similar messages [11591.900508] Lustre: server umount lustre-MDT0000 complete [11596.083532] LustreError: 318892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844627 with bad export cookie 13063368929361458673 [11596.085677] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11596.102676] LustreError: 318892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11596.510223] Lustre: server umount lustre-MDT0001 complete [11600.813640] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11602.724648] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 01:17:11 (1788844631) [11610.882634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11611.544124] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11616.739588] LustreError: 321210:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11616.754856] LustreError: 321210:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11617.348713] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11620.844600] LustreError: 321209:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11625.957899] LustreError: 321210:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11628.679173] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11629.279854] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11635.681090] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11645.507841] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11653.052846] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11662.364277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11662.974736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11671.756402] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11680.421487] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11687.732070] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11687.890858] Lustre: lustre-OST0001: new disk, initializing [11687.893815] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11687.898993] Lustre: 324047:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11687.994886] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11695.305679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11695.316935] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11695.414387] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11695.890335] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11696.498254] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11696.514935] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11697.508976] LustreError: 324362:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11699.557224] LustreError: 324362:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11699.570220] LustreError: 324362:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11703.732532] LustreError: 324362:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11703.749462] LustreError: 324362:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11707.507337] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11707.787952] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11712.039122] LustreError: 324362:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11712.056859] LustreError: 324362:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11714.982520] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9335c36a7000.ost_server_uuid 50 [11717.219757] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9335c36a7000.ost_server_uuid in FULL state after 0 sec [11724.232651] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9335c36a7000.ost_server_uuid 50 [11726.160025] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9335c36a7000.ost_server_uuid in IDLE state after 0 sec [11728.504119] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11744.227845] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11744.228506] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11744.249814] LustreError: Skipped 1 previous similar message [11744.276883] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11744.286212] Lustre: Skipped 4 previous similar messages [11749.342611] Lustre: server umount lustre-OST0001 complete [11751.918212] LustreError: 323031:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11751.961072] LustreError: 323031:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11757.063362] LustreError: 325668:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11757.115435] LustreError: 325668:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11762.149289] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11762.163402] Lustre: Skipped 2 previous similar messages [11762.166459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11762.175757] Lustre: Skipped 4 previous similar messages [11765.250747] Lustre: server umount lustre-OST0000 complete [11772.387258] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11772.417760] Lustre: Skipped 1 previous similar message [11784.161463] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11784.374812] Lustre: server umount lustre-MDT0000 complete [11787.746353] LustreError: 321205:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11787.778256] LustreError: 321205:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11788.616890] LustreError: 321190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844819 with bad export cookie 13063368929361459548 [11788.632863] LustreError: 321190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11788.633166] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11789.108315] Lustre: server umount lustre-MDT0001 complete [11801.678820] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [11804.484196] Key type lgssc unregistered [11804.696641] LNet: 326741:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11804.701741] LNetError: 326741:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11804.720462] LNet: Removed LNI 192.168.202.155@tcp [11805.523298] Key type .llcrypt unregistered [11805.527873] Key type ._llcrypt unregistered [11832.585653] Key type ._llcrypt registered [11832.591421] Key type .llcrypt registered [11832.748164] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [11833.887084] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11833.901488] alg: No test for adler32 (adler32-zlib) [11835.243316] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [11835.480461] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [11837.200193] Key type lgssc registered [11838.455905] Lustre: Echo OBD driver; http://www.lustre.org/ [11846.386878] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11846.409305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11847.964524] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11853.384479] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11858.411820] LustreError: 328157:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11858.434755] LustreError: 328157:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11862.005949] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11863.533093] LustreError: 328156:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11868.644941] LustreError: 328157:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11871.734326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11872.132246] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11877.296430] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11888.901961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11889.463359] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11893.580431] LustreError: 329838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11893.663276] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11897.713017] LustreError: 329838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11897.741482] LustreError: 329838:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11898.413845] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11907.782409] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11908.077501] LustreError: 329841:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11908.096027] LustreError: 329841:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11916.883415] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11917.226469] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11922.482310] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11925.485412] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11935.007826] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11943.854279] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11944.233370] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12055.532075] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12055.555455] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12055.608478] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12055.617945] Lustre: Skipped 1 previous similar message [12060.653813] LustreError: 331079:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12060.678885] LustreError: 331079:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12060.771585] Lustre: server umount lustre-OST0001 complete [12070.886171] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12070.899356] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12070.909076] Lustre: Skipped 2 previous similar messages [12070.924982] Lustre: Skipped 1 previous similar message [12076.048977] Lustre: server umount lustre-OST0000 complete [12081.129332] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12081.142556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12081.167486] Lustre: Skipped 1 previous similar message [12081.183824] Lustre: Skipped 1 previous similar message [12086.244397] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12086.251610] Lustre: Skipped 1 previous similar message [12086.800922] Lustre: server umount lustre-MDT0000 complete [12090.997816] LustreError: 328136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845122 with bad export cookie 9994909860653260652 [12091.002771] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12091.018100] LustreError: 328136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12091.689918] Lustre: server umount lustre-MDT0001 complete [12104.665601] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [12107.753831] Key type lgssc unregistered [12108.125784] LNet: 333105:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12108.137176] LNetError: 333105:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12108.160932] LNet: Removed LNI 192.168.202.155@tcp [12109.222806] Key type .llcrypt unregistered [12109.225801] Key type ._llcrypt unregistered [12159.493401] Key type ._llcrypt registered [12159.494774] Key type .llcrypt registered [12159.627534] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12160.885557] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12160.904400] alg: No test for adler32 (adler32-zlib) [12161.878825] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [12162.065165] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [12163.704409] Key type lgssc registered [12164.752672] Lustre: Echo OBD driver; http://www.lustre.org/ [12174.309185] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12174.336290] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12175.988541] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12181.476253] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12186.600870] LustreError: 335393:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12186.622396] LustreError: 335393:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12190.588273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12190.877024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12195.730263] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12198.749768] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12205.935058] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12206.478533] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12213.377650] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12216.760030] LustreError: 336882:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12220.862578] LustreError: 336882:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12220.874821] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12220.892867] LustreError: 336882:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12224.157309] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12224.517428] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12229.652857] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12233.862781] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12243.850601] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12250.143263] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12267.492885] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12267.502787] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12267.519318] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12270.565585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12270.581401] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12270.607467] Lustre: Skipped 1 previous similar message [12272.484893] Lustre: server umount lustre-MDT0000 complete [12275.704488] LustreError: 335389:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12275.739327] LustreError: 335389:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12277.244081] LustreError: 336530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845308 with bad export cookie 18332447514504720113 [12277.255886] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12277.267930] LustreError: 336530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12278.076305] Lustre: server umount lustre-MDT0001 complete [12294.352266] Lustre: server umount lustre-OST0000 complete [12297.185035] Lustre: 334655:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788845312/real 1788845312] req@ffff92d394ae5c00 x1875740137161856/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788845328 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12297.226394] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12297.239475] Lustre: Skipped 2 previous similar messages [12299.334738] Lustre: server umount lustre-OST0001 complete [12319.695833] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12331.875085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12332.377463] LustreError: 340657:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12332.446513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12337.317901] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12342.757313] LustreError: 340658:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12342.777573] LustreError: 340658:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12347.559319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12347.874817] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12347.879737] Lustre: Skipped 1 previous similar message [12348.052322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12353.644508] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12356.956914] Lustre: 341796:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12364.849109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12365.181076] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12373.382691] LustreError: 342151:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12374.405272] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12377.550483] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12386.248316] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12391.936267] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12394.021563] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12402.348071] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12406.410780] Lustre: 343671:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12412.388631] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12412.397729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12412.412394] Lustre: Skipped 1 previous similar message [12412.437068] Lustre: Skipped 3 previous similar messages [12416.388362] Lustre: server umount lustre-MDT0000 complete [12417.530641] LustreError: 340657:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12417.560607] LustreError: 340657:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [12421.636036] LustreError: 342982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845452 with bad export cookie 18332447514504721814 [12421.643291] LustreError: 342982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12421.643650] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12422.277393] Lustre: server umount lustre-MDT0001 complete [12437.984353] Lustre: 334653:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788845453/real 1788845453] req@ffff92d38df62d80 x1875740137214720/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788845469 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12438.032634] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12438.063924] Lustre: Skipped 2 previous similar messages [12439.258903] Lustre: server umount lustre-OST0000 complete [12441.760427] Lustre: 334656:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788845457/real 1788845457] req@ffff92d384308a80 x1875740137214976/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788845473 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12444.490508] Lustre: server umount lustre-OST0001 complete [12448.040253] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12449.918643] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12451.466259] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 01:31:20 (1788845480) [12474.279661] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12482.813121] loop0: detected capacity change from 0 to 1099511627776 [12483.626321] SGI XFS with ACLs, security attributes, quota, no debug enabled [12483.641045] *** VALIDATE xfs *** [12483.662807] XFS (loop0): Mounting V5 Filesystem [12483.719969] XFS (loop0): Ending clean mount [12485.277142] loop1: detected capacity change from 0 to 18691697672192 [12627.916909] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [12638.753754] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12678.742115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12679.084346] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12679.146792] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12679.285452] Lustre: lustre-MDT0000: new disk, initializing [12679.383934] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12679.396686] Lustre: Skipped 1 previous similar message [12679.407249] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12684.058776] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12697.096587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12697.278807] Lustre: 348814:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12697.349873] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12697.354616] Lustre: Skipped 1 previous similar message [12697.475443] Lustre: lustre-MDT0001: new disk, initializing [12697.733163] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12697.745818] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12703.578543] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12715.456367] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12722.246563] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12730.087544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12730.434566] Lustre: lustre-OST0000: new disk, initializing [12730.438611] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12730.452573] Lustre: 350787:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12731.974821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12731.982554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12732.126775] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12738.505163] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12749.800658] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12755.821471] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12756.121595] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12760.963823] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12761.331597] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12768.228995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12768.236337] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12768.238914] Lustre: Skipped 1 previous similar message [12768.242827] LustreError: Skipped 1 previous similar message [12768.243737] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12768.260087] Lustre: Skipped 1 previous similar message [12769.971544] Lustre: server umount lustre-OST0000 complete [12774.370643] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12778.470673] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12778.486397] Lustre: Skipped 2 previous similar messages [12780.308991] Lustre: server umount lustre-MDT0000 complete [12783.585641] LustreError: 348827:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12783.612751] LustreError: 348827:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12784.144193] LustreError: 348807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845815 with bad export cookie 18332447514504723361 [12784.146813] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12784.152916] LustreError: 348807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12784.557785] Lustre: server umount lustre-MDT0001 complete [12791.926590] XFS (loop0): Unmounting Filesystem [12796.495152] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 01:37:05 (1788845825) [12803.492125] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12804.132068] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12804.135023] Lustre: Skipped 2 previous similar messages [12808.812414] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12817.452815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12823.110926] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12831.744564] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12838.966495] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12846.934458] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12854.957585] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12863.652722] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12875.233336] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12875.238065] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12875.244662] Lustre: Skipped 2 previous similar messages [12875.247656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12878.844608] Lustre: server umount lustre-OST0000 complete [12884.450080] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12884.456827] LustreError: Skipped 1 previous similar message [12889.530694] Lustre: server umount lustre-MDT0000 complete [12893.545958] LustreError: 355193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845924 with bad export cookie 18332447514504724243 [12893.551650] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12893.553196] LustreError: 355193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12893.857598] Lustre: server umount lustre-MDT0001 complete [12904.608488] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [12907.282942] Key type lgssc unregistered [12907.675857] LNet: 356977:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12907.697338] LNetError: 356977:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12907.726247] LNet: Removed LNI 192.168.202.155@tcp [12909.011324] Key type .llcrypt unregistered [12909.014765] Key type ._llcrypt unregistered [12935.413667] Key type ._llcrypt registered [12935.415503] Key type .llcrypt registered [12935.455485] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12936.204823] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12936.239214] alg: No test for adler32 (adler32-zlib) [12937.396791] Lustre: Lustre: Build Version: 2.17.57_106_g45f4cc0 [12937.588226] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [12939.248228] Key type lgssc registered [12940.374539] Lustre: Echo OBD driver; http://www.lustre.org/ [12948.319568] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12948.341232] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12949.990617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12955.294507] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12960.230174] LustreError: 358542:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12960.266258] LustreError: 358542:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12965.356382] LustreError: 358541:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12965.431924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12966.115929] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12972.338443] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12981.884108] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12989.159226] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12997.719176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12998.146898] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13005.092244] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13013.385303] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13032.946083] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13032.965091] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13032.997406] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13035.134294] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [13036.009555] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13040.207632] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [13040.221765] Lustre: Skipped 2 previous similar messages [13045.311194] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [13045.315516] Lustre: Skipped 2 previous similar messages [13046.245388] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13046.482544] Lustre: server umount lustre-MDT0001 complete [13050.442293] LustreError: 360381:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13055.551647] LustreError: 358536:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13055.592114] LustreError: 358536:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13059.094827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13059.318503] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13059.328329] Lustre: Found index 1 for lustre-MDT0001, updating log [13059.365198] Lustre: 358529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13059.787413] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13065.200208] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13065.269237] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13068.297473] LustreError: 362166:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13068.313418] LustreError: 362166:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13068.329593] LustreError: 358533:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13068.457180] LustreError: 362169:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -17 [13068.470071] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13068.470071] [13372.385452] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13372.388177] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13372.407856] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13372.416618] Lustre: Skipped 3 previous similar messages [13378.316947] Lustre: server umount lustre-MDT0001 complete [13382.627894] LustreError: 359975:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13382.654243] LustreError: 359975:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13392.528659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13392.715247] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13392.719169] Lustre: Found index 1 for lustre-MDT0001, updating log [13392.756611] Lustre: 358531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13392.875435] LustreError: 359975:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13392.903911] LustreError: 359975:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [13393.424492] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13398.525246] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13400.206986] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13401.630345] LustreError: 370027:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13401.643540] LustreError: 370027:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13401.667651] LustreError: 358533:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13401.677419] LustreError: 358533:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13401.807996] LustreError: 370044:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13401.820438] LustreError: 370044:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13401.837115] LustreError: 370044:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -17 [13401.854374] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.155@tcp [13401.854374] [13710.820541] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13710.834599] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13710.838356] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13710.858928] Lustre: Skipped 2 previous similar messages [13710.865752] Lustre: Skipped 4 previous similar messages [13712.846734] Lustre: server umount lustre-MDT0001 complete [13715.942053] LustreError: 358541:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13715.986667] LustreError: 358541:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13723.728283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13723.804374] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13723.814561] Lustre: Found index 1 for lustre-MDT0001, updating log [13723.846267] Lustre: 358531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13724.154722] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13729.265457] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13729.308611] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13730.821964] LustreError: 378038:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13730.826526] LustreError: 378038:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13730.846936] LustreError: 358533:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13730.857872] LustreError: 358533:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13730.969791] LustreError: 378047:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13730.985385] LustreError: 378047:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -114 [13730.992875] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13730.992875] [14056.947686] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14056.953133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14056.957537] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14056.964995] Lustre: Skipped 2 previous similar messages [14056.992341] Lustre: Skipped 4 previous similar messages [14060.861928] Lustre: server umount lustre-MDT0000 complete [14062.064512] LustreError: 358538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14062.094434] LustreError: 358538:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [14064.884772] LustreError: 358523:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847096 with bad export cookie 1943998898025682088 [14064.915961] LustreError: 358523:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14064.926583] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14067.170104] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14071.416926] Lustre: server umount lustre-MDT0001 complete [14078.212180] Lustre: server umount lustre-OST0000 complete [14091.204561] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [14103.002644] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [14145.606661] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 01:59:35 (1788847175) [14153.209037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14153.458918] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14153.485412] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14153.558517] Lustre: lustre-MDT0000: new disk, initializing [14153.616586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14153.641923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14158.780592] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [14171.593838] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14171.713142] Lustre: 388250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14171.772712] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14171.776672] Lustre: Skipped 1 previous similar message [14171.857633] Lustre: lustre-MDT0001: new disk, initializing [14171.962191] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14172.013691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14172.026894] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14177.478182] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [14187.547454] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14194.011948] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14200.892124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14201.190391] Lustre: lustre-OST0000: new disk, initializing [14201.193875] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14201.199696] Lustre: 390217:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14201.272443] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14207.548415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14207.557475] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14207.674774] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14208.491167] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [14219.514149] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14228.464223] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14228.465970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14228.477384] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14228.494059] Lustre: Skipped 2 previous similar messages [14228.543996] Lustre: Skipped 4 previous similar messages [14229.331236] Lustre: server umount lustre-MDT0000 complete [14233.574865] LustreError: 388255:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14233.608794] LustreError: 388255:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [14234.064890] LustreError: 388240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847265 with bad export cookie 1943998898025684041 [14234.071188] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14234.071598] LustreError: 388240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14234.084060] LustreError: 389055:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14234.090982] LustreError: 389055:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000813cb316 x1875740950142464/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [14234.503027] Lustre: server umount lustre-MDT0001 complete [14249.278263] Lustre: server umount lustre-OST0000 complete [14281.920646] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [14292.226536] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [14329.055599] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14041 sec ========= 02:02:37 (1788847357) [14331.562714] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 02:02:40 (1788847360) === [14335.796813] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 02:02:44 (1788847364) === [14381.671456] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [14385.723710] Key type lgssc unregistered [14386.114732] LNet: 396597:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14386.130657] LNetError: 396597:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14386.149383] LNet: Removed LNI 192.168.202.155@tcp [14387.080397] Key type .llcrypt unregistered [14387.083105] Key type ._llcrypt unregistered