[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 473195501 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003150] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008487] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010013] pid_max: default: 32768 minimum: 301 [ 0.012074] LSM: Security Framework initializing [ 0.013060] Yama: becoming mindful. [ 0.014040] SELinux: Initializing. [ 0.015071] *** VALIDATE selinux *** [ 0.022820] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028087] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029165] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030106] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031113] *** VALIDATE tmpfs *** [ 0.032457] *** VALIDATE proc *** [ 0.033271] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.035282] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037015] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039029] Spectre V2 : User space: Vulnerable [ 0.040008] Speculative Store Bypass: Vulnerable [ 0.043193] debug: unmapping init [mem 0xffffffff87859000-0xffffffff87860fff] [ 0.045201] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046678] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047023] ... version: 2 [ 0.048016] ... bit width: 48 [ 0.049013] ... generic registers: 4 [ 0.050014] ... value mask: 0000ffffffffffff [ 0.051057] ... max period: 00007fffffffffff [ 0.052013] ... fixed-purpose events: 3 [ 0.053009] ... event mask: 000000070000000f [ 0.055201] rcu: Hierarchical SRCU implementation. [ 0.057407] smp: Bringing up secondary CPUs ... [ 0.058619] x86: Booting SMP configuration: [ 0.059028] .... node #0, CPUs: #1 #2 #3 [ 0.062477] smp: Brought up 1 node, 4 CPUs [ 0.064016] smpboot: Max logical packages: 1 [ 0.065023] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.260176] node 0 deferred pages initialised in 193ms [ 0.264385] devtmpfs: initialized [ 0.265245] x86/mm: Memory block size: 128MB [ 0.267737] gcov: version magic: 0x41383552 [ 0.268609] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.273175] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.276540] pinctrl core: initialized pinctrl subsystem [ 0.278232] [ 0.278866] ************************************************************* [ 0.282020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.284017] ** ** [ 0.286016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.289021] ** ** [ 0.291017] ** This means that this kernel is built to expose internal ** [ 0.294020] ** IOMMU data structures, which may compromise security on ** [ 0.296026] ** your system. ** [ 0.298020] ** ** [ 0.300018] ** If you see this message and you are not debugging the ** [ 0.303019] ** kernel, report this immediately to your vendor! ** [ 0.305015] ** ** [ 0.308016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.310018] ************************************************************* [ 0.313816] NET: Registered protocol family 16 [ 0.315508] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.318099] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.321080] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.324111] cpuidle: using governor menu [ 0.325688] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.328694] PCI: Using configuration type 1 for base access [ 0.331154] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.340162] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.342042] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.345141] cryptd: max_cpu_qlen set to 1000 [ 0.347292] ACPI: Added _OSI(Module Device) [ 0.349055] ACPI: Added _OSI(Processor Device) [ 0.350026] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.352021] ACPI: Added _OSI(Processor Aggregator Device) [ 0.356893] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.364600] ACPI: Interpreter enabled [ 0.366077] ACPI: PM: (supports S0 S3 S4 S5) [ 0.367013] ACPI: Using IOAPIC for interrupt routing [ 0.368181] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.372466] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.382891] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.386070] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.389027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.393109] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.398568] acpiphp: Slot [2] registered [ 0.400208] acpiphp: Slot [5] registered [ 0.401180] acpiphp: Slot [6] registered [ 0.404198] acpiphp: Slot [7] registered [ 0.406165] acpiphp: Slot [8] registered [ 0.407096] acpiphp: Slot [9] registered [ 0.408135] acpiphp: Slot [10] registered [ 0.410152] acpiphp: Slot [3] registered [ 0.412170] acpiphp: Slot [4] registered [ 0.414136] acpiphp: Slot [11] registered [ 0.416141] acpiphp: Slot [12] registered [ 0.418145] acpiphp: Slot [13] registered [ 0.420145] acpiphp: Slot [14] registered [ 0.422160] acpiphp: Slot [15] registered [ 0.424189] acpiphp: Slot [16] registered [ 0.427192] acpiphp: Slot [17] registered [ 0.428153] acpiphp: Slot [18] registered [ 0.430107] acpiphp: Slot [19] registered [ 0.431121] acpiphp: Slot [20] registered [ 0.432136] acpiphp: Slot [21] registered [ 0.434118] acpiphp: Slot [22] registered [ 0.435167] acpiphp: Slot [23] registered [ 0.436116] acpiphp: Slot [24] registered [ 0.438127] acpiphp: Slot [25] registered [ 0.440139] acpiphp: Slot [26] registered [ 0.441132] acpiphp: Slot [27] registered [ 0.443130] acpiphp: Slot [28] registered [ 0.444159] acpiphp: Slot [29] registered [ 0.446117] acpiphp: Slot [30] registered [ 0.447132] acpiphp: Slot [31] registered [ 0.449120] PCI host bridge to bus 0000:00 [ 0.450022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.452033] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.455028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.457040] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.461037] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.464030] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.466258] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.470091] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.473640] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.484792] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.488000] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.493039] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.496023] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.499023] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.503090] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.505891] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.509405] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.512911] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.519021] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.532019] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.536017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.542395] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.550021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.560021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.585023] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.595352] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.602017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.614018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.633017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.648146] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.654023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.661024] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.682020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.690215] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.698014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.710017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.731019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.743138] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.749018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.756019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.774022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.783000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.789032] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.794019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.807022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.818362] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.821422] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.824523] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.827443] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.829237] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.833638] iommu: Default domain type: Passthrough [ 0.836509] SCSI subsystem initialized [ 0.837158] ACPI: bus type USB registered [ 0.839155] usbcore: registered new interface driver usbfs [ 0.841111] usbcore: registered new interface driver hub [ 0.843116] usbcore: registered new device driver usb [ 0.847235] pps_core: LinuxPPS API ver. 1 registered [ 0.849014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.852081] PTP clock support registered [ 0.855047] EDAC MC: Ver: 3.0.0 [ 0.857170] PCI: Using ACPI for IRQ routing [ 0.858672] NetLabel: Initializing [ 0.860020] NetLabel: domain hash size = 128 [ 0.862015] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.864108] NetLabel: unlabeled traffic allowed by default [ 0.867053] vgaarb: loaded [ 0.868318] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.870015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.875347] clocksource: Switched to clocksource kvm-clock [ 0.983141] VFS: Disk quotas dquot_6.6.0 [ 0.984964] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.987809] *** VALIDATE ramfs *** [ 0.989291] *** VALIDATE hugetlbfs *** [ 0.991106] pnp: PnP ACPI init [ 0.993056] pnp: PnP ACPI: found 6 devices [ 1.009238] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.012337] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.014342] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.016197] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.018539] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.020867] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.023457] NET: Registered protocol family 2 [ 1.025674] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.030450] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.033771] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.038747] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.041953] TCP: Hash tables configured (established 65536 bind 65536) [ 1.044759] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.049041] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.051709] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.054321] NET: Registered protocol family 1 [ 1.057451] RPC: Registered named UNIX socket transport module. [ 1.060018] RPC: Registered udp transport module. [ 1.061878] RPC: Registered tcp transport module. [ 1.063966] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.066681] NET: Registered protocol family 44 [ 1.068617] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.070956] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.073217] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.075792] PCI: CLS 0 bytes, default 64 [ 1.077572] Unpacking initramfs... [ 2.528650] debug: unmapping init [mem 0xffff9590fcc54000-0xffff9590fffbffff] [ 2.535879] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.538091] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.541970] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.030215] Initialise system trusted keyrings [ 3.031493] Key type blacklist registered [ 3.033035] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.042545] zbud: loaded [ 3.045740] *** VALIDATE nfs *** [ 3.047948] *** VALIDATE nfs4 *** [ 3.049273] pstore: using deflate compression [ 3.052117] Platform Keyring initialized [ 3.157077] NET: Registered protocol family 38 [ 3.159352] Key type asymmetric registered [ 3.161318] Asymmetric key parser 'x509' registered [ 3.163554] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.167184] io scheduler mq-deadline registered [ 3.168772] io scheduler kyber registered [ 3.170651] io scheduler bfq registered [ 3.173177] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.175831] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.178717] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.181272] ACPI: Power Button [PWRF] [ 3.186366] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.192556] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.215418] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.226143] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.250567] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.277905] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.307123] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.313479] Non-volatile memory driver v1.3 [ 3.315756] Linux agpgart interface v0.103 [ 3.349529] virtio_blk virtio1: [vda] 145808 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.353128] vda: detected capacity change from 0 to 74653696 [ 3.369603] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.373900] vdb: detected capacity change from 0 to 1073741824 [ 3.390245] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.393317] vdc: detected capacity change from 0 to 2621440000 [ 3.413461] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.418086] vdd: detected capacity change from 0 to 2621440000 [ 3.444752] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.448263] vde: detected capacity change from 0 to 4294967296 [ 3.461764] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.464662] vdf: detected capacity change from 0 to 4294967296 [ 3.482769] libphy: Fixed MDIO Bus: probed [ 3.492968] usbcore: registered new interface driver usbserial_generic [ 3.494853] usbserial: USB Serial support registered for generic [ 3.496982] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.500797] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.502678] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.505197] mousedev: PS/2 mouse device common for all mice [ 3.508379] rtc_cmos 00:05: RTC can wake from S4 [ 3.508788] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.512295] rtc_cmos 00:05: registered as rtc0 [ 3.517044] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.521141] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.522792] intel_pstate: CPU model not supported [ 3.528616] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.529905] hid: raw HID events driver (C) Jiri Kosina [ 3.536020] usbcore: registered new interface driver usbhid [ 3.538195] usbhid: USB HID core driver [ 3.539962] drop_monitor: Initializing network drop monitor service [ 3.542629] Initializing XFRM netlink socket [ 3.544914] NET: Registered protocol family 10 [ 3.548392] Segment Routing with IPv6 [ 3.550038] NET: Registered protocol family 17 [ 3.552485] mpls_gso: MPLS GSO support [ 3.558590] RAS: Correctable Errors collector initialized. [ 3.560574] AVX version of gcm_enc/dec engaged. [ 3.562051] AES CTR mode by8 optimization enabled [ 3.640569] sched_clock: Marking stable (3640530483, 0)->(4576865076, -936334593) [ 3.644610] registered taskstats version 1 [ 3.646981] Loading compiled-in X.509 certificates [ 3.649237] zswap: loaded using pool lzo/zbud [ 3.675527] Key type big_key registered [ 3.688749] Key type encrypted registered [ 3.690730] ima: No TPM chip found, activating TPM-bypass! [ 3.692634] ima: Allocated hash algorithm: sha1 [ 3.693806] ima: No architecture policies found [ 3.694964] evm: Initialising EVM extended attributes: [ 3.696097] evm: security.selinux [ 3.696892] evm: security.ima [ 3.697655] evm: security.capability [ 3.698638] evm: HMAC attrs: 0x1 [ 3.700704] rtc_cmos 00:05: setting system clock to 2026-07-20 21:42:23 UTC (1784583743) [ 3.704558] debug: unmapping init [mem 0xffffffff88803000-0xffffffff889fffff] [ 3.706564] debug: unmapping init [mem 0xffffffff87582000-0xffffffff87858fff] [ 3.714175] Write protecting the kernel read-only data: 28672k [ 3.717113] debug: unmapping init [mem 0xffffffff85c03000-0xffffffff85dfffff] [ 3.719233] debug: unmapping init [mem 0xffffffff86514000-0xffffffff865fffff] [ 3.752657] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.759306] systemd[1]: Detected virtualization kvm. [ 3.760719] systemd[1]: Detected architecture x86-64. [ 3.762126] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.783485] systemd[1]: No hostname configured. [ 3.784877] systemd[1]: Set hostname to . [ 3.786547] random: systemd: uninitialized urandom read (16 bytes read) [ 3.788323] systemd[1]: Initializing machine ID from random generator. [ 3.839503] random: ln: uninitialized urandom read (6 bytes read) [ 3.918360] random: systemd: uninitialized urandom read (16 bytes read) [ 3.920667] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.923615] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.926450] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.524861] device-mapper: uevent: version 1.0.3 [ 4.526787] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.247963] random: fast init done [ 5.297946] scsi host0: ata_piix [ 5.309615] virtio_net virtio0 ens2: renamed from eth0 [ 5.323483] scsi host1: ata_piix [ 5.324797] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.326381] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.093186] dracut-initqueue[583]: RTNETLINK answers: File exists [ 10.203265] random: crng init done [ 10.204375] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.612861] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.684039] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.911640] SELinux: Disabled at runtime. [ 11.971488] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.976808] systemd[1]: Detected virtualization kvm. [ 11.977880] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.456282] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.459209] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.462729] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.465527] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.467745] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.474272] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.478976] systemd[1]: Created slice system-getty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting udev Coldplug all Devices... Mounting POSIX Message Queue File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... [ 12.609961] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.921718] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.300378] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.347744] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.482238] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.498419] EDAC sbridge: Ver: 1.1.2 [ 15.120602] Key type dns_resolver registered [ 15.427123] NFS: Registering the id_resolver key type [ 15.430279] Key type id_resolver registered [ 15.431404] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Authorization Manager. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg335-server login: [ 78.651564] libcfs: loading out-of-tree module taints kernel. [ 78.724651] Key type ._llcrypt registered [ 78.728373] Key type .llcrypt registered [ 78.825501] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 95.853982] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 97.402186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 97.413898] alg: No test for adler32 (adler32-zlib) [ 98.846639] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 99.699977] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 101.513293] Key type lgssc registered [ 103.411296] Lustre: Echo OBD driver; http://www.lustre.org/ [ 120.673247] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 166.940511] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 182.209925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 182.262076] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 183.639955] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 183.696736] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 183.811262] Lustre: lustre-MDT0000: new disk, initializing [ 183.900171] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 183.920235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 188.376573] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 201.016924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 201.142484] Lustre: 6531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 201.171895] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 201.179389] Lustre: Skipped 1 previous similar message [ 201.247376] Lustre: lustre-MDT0001: new disk, initializing [ 201.304576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 201.340723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 201.351467] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 206.451651] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 212.445645] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 224.156063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 224.404681] Lustre: lustre-OST0000: new disk, initializing [ 224.406611] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 224.411439] Lustre: 8468:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 224.458517] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 227.378938] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 227.395957] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 227.542849] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 229.790616] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 243.047561] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 243.197188] Lustre: lustre-OST0001: new disk, initializing [ 243.199646] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 243.204891] Lustre: 9541:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 243.268801] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 248.730530] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 251.429952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 251.439842] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 251.507067] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 259.059680] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 263.900594] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 268.375534] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing check_logdir /tmp/testlogs/ [ 274.071674] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing yml_node [ 278.586796] Lustre: DEBUG MARKER: Client: 2.17.54.183 [ 280.737256] Lustre: DEBUG MARKER: MDS: 2.17.54.183 [ 283.122407] Lustre: DEBUG MARKER: OSS: 2.17.54.183 [ 284.338517] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Jul 20 17:47:03 EDT 2026 [ 298.760386] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 299.864285] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 317.920918] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 317.925053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.953237] Lustre: Skipped 3 previous similar messages [ 317.967294] Lustre: Skipped 3 previous similar messages [ 321.721639] Lustre: server umount lustre-MDT0000 complete [ 325.200892] LustreError: 6521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584064 with bad export cookie 14185422608683166655 [ 325.210032] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 325.228249] LustreError: 6521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 328.161663] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 328.166704] LustreError: 8648:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 328.173876] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 328.207338] LustreError: 8648:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 330.210066] LustreError: 6538:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.210140] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 330.226492] LustreError: 6538:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 330.231434] Lustre: Skipped 2 previous similar messages [ 331.617636] Lustre: server umount lustre-MDT0001 complete [ 335.382236] Lustre: server umount lustre-OST0000 complete [ 336.818294] hrtimer: interrupt took 7287375 ns [ 338.741923] Lustre: server umount lustre-OST0001 complete [ 344.589315] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 352.773667] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 383.303093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 383.522801] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 383.579712] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 383.702645] Lustre: lustre-MDT0000: new disk, initializing [ 383.806725] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 383.838604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 387.467209] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 396.022783] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 396.146178] Lustre: 15087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 396.219097] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 396.225574] Lustre: Skipped 1 previous similar message [ 396.366402] Lustre: lustre-MDT0001: new disk, initializing [ 396.440631] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 396.487673] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 396.499090] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 400.584980] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 410.329606] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 415.858180] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 422.458023] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 422.772917] Lustre: lustre-OST0000: new disk, initializing [ 422.777124] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 422.783793] Lustre: 17059:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 422.918332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 424.052882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 424.069726] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 424.144463] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 428.301420] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 436.959364] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 441.757777] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 442.043990] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 446.851183] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 447.078757] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 450.016939] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 450.027396] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 450.032170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 450.053214] Lustre: Skipped 2 previous similar messages [ 450.065091] Lustre: Skipped 1 previous similar message [ 455.136780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 455.149537] Lustre: Skipped 1 previous similar message [ 463.328114] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 463.511660] Lustre: server umount lustre-OST0000 complete [ 468.455333] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 468.467648] LustreError: Skipped 1 previous similar message [ 468.476979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 468.490745] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 468.496178] Lustre: Skipped 2 previous similar messages [ 475.469563] Lustre: server umount lustre-MDT0000 complete [ 475.616187] LustreError: 15100:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 475.638221] LustreError: 15100:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 478.669523] LustreError: 15080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584218 with bad export cookie 14185422608683168181 [ 478.680183] LustreError: 15080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 478.682199] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 479.014920] Lustre: server umount lustre-MDT0001 complete [ 483.681759] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 17:50:22 (1784584222) [ 489.884257] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 490.413332] LustreError: 19324:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 490.475792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 494.212981] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 495.598716] LustreError: 19325:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 502.386926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 507.269131] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 514.386068] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 519.813970] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 525.759696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 530.985376] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 536.891176] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 540.096502] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 545.766960] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 545.768463] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 545.788322] Lustre: Skipped 3 previous similar messages [ 545.795960] Lustre: Skipped 3 previous similar messages [ 551.140500] Lustre: server umount lustre-MDT0000 complete [ 554.061902] LustreError: 19306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584293 with bad export cookie 14185422608683169070 [ 554.066236] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 554.068375] LustreError: 19306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 554.275277] Lustre: server umount lustre-MDT0001 complete [ 567.737503] Lustre: server umount lustre-OST0000 complete [ 585.412968] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 594.294608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 594.600522] LustreError: 23703:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 594.623651] LustreError: 23703:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 594.664252] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 594.667650] Lustre: Skipped 2 previous similar messages [ 598.522747] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 606.280902] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 611.183196] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 613.909202] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 619.732617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 620.013935] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 620.016143] Lustre: Skipped 3 previous similar messages [ 626.358890] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 633.639843] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 633.814978] Lustre: lustre-OST0001: new disk, initializing [ 633.819177] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 633.830783] Lustre: 26018:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 635.881492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 635.895904] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 635.968375] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 639.590985] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 647.992193] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 651.023485] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 663.011518] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 663.027205] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 668.230275] Lustre: server umount lustre-MDT0000 complete [ 671.559952] LustreError: 23683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584411 with bad export cookie 14185422608683169917 [ 671.586118] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 671.589263] LustreError: 23683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 671.716692] LustreError: 26551:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 671.742182] LustreError: 26551:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 672.014354] Lustre: server umount lustre-MDT0001 complete [ 678.447473] Lustre: server umount lustre-OST0000 complete [ 681.940946] Lustre: server umount lustre-OST0001 complete [ 697.292059] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 705.937856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 706.376466] LustreError: 29039:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 710.557572] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 718.935158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 723.925606] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 726.890970] Lustre: 30186:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 732.486967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 733.041384] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 733.046950] Lustre: Skipped 5 previous similar messages [ 738.866973] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 746.692620] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 752.711413] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 759.391690] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 762.175792] Lustre: 32062:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 767.463995] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 767.467672] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 767.486503] Lustre: Skipped 7 previous similar messages [ 767.499870] Lustre: Skipped 10 previous similar messages [ 773.811892] Lustre: server umount lustre-MDT0000 complete [ 777.017837] LustreError: 29019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584516 with bad export cookie 14185422608683171212 [ 777.028872] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 777.030341] LustreError: 29019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 777.321378] Lustre: server umount lustre-MDT0001 complete [ 791.703853] Lustre: server umount lustre-OST0000 complete [ 794.081148] Lustre: 3661:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784584517/real 1784584517] req@ffff959043233b80 x1871271783837696/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784584533 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 795.215397] Lustre: server umount lustre-OST0001 complete [ 802.076272] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 17:55:41 (1784584541) [ 824.228796] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 834.196586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 834.486458] LustreError: 34863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 834.514365] LustreError: 34863:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 838.640323] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 846.055227] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 849.488737] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 851.759272] Lustre: 36011:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 857.641435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 863.902065] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 872.664419] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 878.829739] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 886.232629] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 889.738342] Lustre: 37889:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 897.516623] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 897.525746] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 897.546427] Lustre: Skipped 2 previous similar messages [ 897.555074] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 897.561669] Lustre: Skipped 4 previous similar messages [ 900.740530] Lustre: server umount lustre-MDT0000 complete [ 904.196366] LustreError: 35614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584643 with bad export cookie 14185422608683172318 [ 904.200153] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 904.205157] LustreError: 35614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 904.559864] Lustre: server umount lustre-MDT0001 complete [ 919.063360] Lustre: server umount lustre-OST0000 complete [ 931.483273] Lustre: server umount lustre-OST0001 complete [ 939.557971] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 17:57:58 (1784584678) [ 953.652962] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 961.827713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 967.182482] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 967.654475] LustreError: 40033:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 967.680065] LustreError: 40033:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 975.730348] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 980.630162] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 983.516815] Lustre: 41184:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 989.750972] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 990.038268] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 990.042603] Lustre: Skipped 7 previous similar messages [ 995.298827] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1003.248192] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1008.560256] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1015.180634] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1018.301709] Lustre: 43061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1019.120240] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1019.124528] Lustre: Skipped 1 previous similar message [ 1022.436572] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1022.443617] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1022.470524] Lustre: Skipped 3 previous similar messages [ 1027.713726] LustreError: 43269:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (ffff959044fd0600) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 1027.838079] Lustre: server umount lustre-MDT0000 complete [ 1030.870838] LustreError: 43063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584770 with bad export cookie 14185422608683173424 [ 1030.889864] LustreError: 43063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1030.890812] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1037.311100] Lustre: server umount lustre-MDT0001 complete [ 1044.126566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1055.712264] LustreError: 3660:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9590436c1f80 x1871271783934464/t0(0) o250->MGC192.168.203.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1059.839059] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1065.331743] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1067.471861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1072.192337] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1073.138208] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1073.145843] Lustre: Skipped 1 previous similar message [ 1079.297702] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1080.755187] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1085.858439] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1087.292354] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1093.599882] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1097.692210] Lustre: server umount lustre-MDT0000 complete [ 1101.123224] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584840 with bad export cookie 14185422608683174551 [ 1101.137738] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1101.141110] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1101.418756] Lustre: server umount lustre-MDT0001 complete [ 1114.543490] Lustre: server umount lustre-OST0000 complete [ 1128.348625] Lustre: server umount lustre-OST0001 complete [ 1135.858148] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:01:14 (1784584874) [ 1150.261271] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1160.429546] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1165.229139] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1173.064745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1178.003857] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1180.815667] Lustre: 48834:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1187.719766] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1194.144940] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1203.585339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1211.137466] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1217.826424] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1221.874789] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1234.916466] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1234.938809] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1234.961080] Lustre: Skipped 9 previous similar messages [ 1234.972723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1234.979478] Lustre: Skipped 19 previous similar messages [ 1235.267170] Lustre: server umount lustre-MDT0000 complete [ 1236.963452] LustreError: 48420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1237.003567] LustreError: 48420:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 34 previous similar messages [ 1239.146228] LustreError: 48437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784584978 with bad export cookie 14185422608683175342 [ 1239.167593] LustreError: 48437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1239.168780] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1239.612209] Lustre: server umount lustre-MDT0001 complete [ 1253.883714] Lustre: server umount lustre-OST0000 complete [ 1266.341472] Lustre: server umount lustre-OST0001 complete [ 1274.787677] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:03:33 (1784585013) [ 1276.749222] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1278.282052] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:03:37 (1784585017) [ 1289.610496] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 1299.589679] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1323.123695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1323.399576] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1323.469559] Lustre: lustre-MDT0000: new disk, initializing [ 1323.559960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1327.126658] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1335.018941] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1341.049152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1341.383850] Lustre: lustre-OST0000: new disk, initializing [ 1341.390824] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1341.397448] Lustre: Skipped 1 previous similar message [ 1341.402764] Lustre: 54908:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1342.654211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1342.666733] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1342.730334] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1346.808641] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1353.977560] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1355.901222] Lustre: DEBUG MARKER: create test files [ 1389.025690] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1391.610248] Lustre: server umount lustre-OST0000 complete [ 1394.745857] Lustre: server umount lustre-MDT0000 complete [ 1404.424111] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 1406.969883] Key type lgssc unregistered [ 1407.262061] LNet: 56535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1407.279446] LNetError: 56535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1407.297820] LNet: Removed LNI 192.168.203.135@tcp [ 1408.096153] Key type .llcrypt unregistered [ 1408.099665] Key type ._llcrypt unregistered [ 1434.255492] Key type ._llcrypt registered [ 1434.258503] Key type .llcrypt registered [ 1434.399380] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1436.218946] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1436.253595] alg: No test for adler32 (adler32-zlib) [ 1437.239421] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 1437.430100] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 1439.087162] Key type lgssc registered [ 1440.096693] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1446.342899] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1446.371163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1447.887972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1452.217878] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1458.506578] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1463.690322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1463.985521] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1469.461596] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1473.217639] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1473.319356] Lustre: 59258:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1473.328316] Lustre: 59258:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1473.334592] Lustre: 59258:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1473.346130] Lustre: 59258:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1473.354038] Lustre: 59258:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1473.360297] Lustre: 59258:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1475.710571] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1480.045974] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1484.383694] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1492.133346] Lustre: server umount lustre-OST0000 complete [ 1505.896329] Lustre: server umount lustre-MDT0000 complete [ 1515.458105] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 1518.400962] Key type lgssc unregistered [ 1518.754430] LNet: 60866:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1518.760298] LNetError: 60866:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1518.789973] LNet: Removed LNI 192.168.203.135@tcp [ 1519.801174] Key type .llcrypt unregistered [ 1519.806732] Key type ._llcrypt unregistered [ 1551.614834] Key type ._llcrypt registered [ 1551.620037] Key type .llcrypt registered [ 1551.740837] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1553.126603] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1553.278992] alg: No test for adler32 (adler32-zlib) [ 1554.406912] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 1554.631308] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 1556.263314] Key type lgssc registered [ 1557.253720] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1564.608621] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1564.638218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1566.245876] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1571.241357] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1577.897842] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1584.227322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1584.524480] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1590.143440] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1590.731036] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1590.808358] Lustre: 63706:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1590.822269] Lustre: 63706:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1590.829574] Lustre: 63706:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1590.836520] Lustre: 63706:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1590.847361] Lustre: 63706:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1590.862841] Lustre: 63706:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1596.781971] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1601.838428] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1611.234994] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1611.244588] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1611.268097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1616.354113] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1616.435909] Lustre: server umount lustre-OST0000 complete [ 1621.131608] Lustre: server umount lustre-MDT0000 complete [ 1631.566523] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 1633.821693] Key type lgssc unregistered [ 1634.062593] LNet: 65263:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1634.065983] LNetError: 65263:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1635.111036] LNet: Removed LNI 192.168.203.135@tcp [ 1635.868123] Key type .llcrypt unregistered [ 1635.870276] Key type ._llcrypt unregistered [ 1652.834923] Key type ._llcrypt registered [ 1652.842576] Key type .llcrypt registered [ 1652.997550] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 1668.870351] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1669.947915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1669.964338] alg: No test for adler32 (adler32-zlib) [ 1670.996574] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 1671.198358] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 1672.871187] Key type lgssc registered [ 1673.900767] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1704.159810] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1704.186305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1705.432578] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1705.503063] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1705.591976] Lustre: lustre-MDT0000: new disk, initializing [ 1705.678657] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1705.696750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1710.087548] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1719.481085] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1719.666658] Lustre: 68418:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1719.752606] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1719.760417] Lustre: Skipped 1 previous similar message [ 1719.872885] Lustre: lustre-MDT0001: new disk, initializing [ 1719.954238] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1720.009536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1720.025544] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1724.816611] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1732.997351] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1739.522916] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1745.655148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1745.969590] Lustre: lustre-OST0000: new disk, initializing [ 1745.975955] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1745.990343] Lustre: 70387:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1746.138109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1751.489656] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1755.694344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1755.710344] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1755.826629] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1761.232528] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1765.981782] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1766.295633] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1770.640872] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1770.832340] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1776.610843] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1776.615722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1776.627320] LustreError: Skipped 1 previous similar message [ 1776.627921] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1776.655453] Lustre: Skipped 1 previous similar message [ 1778.725753] Lustre: server umount lustre-OST0000 complete [ 1786.848686] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1786.854424] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1786.867121] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1786.903965] Lustre: Skipped 1 previous similar message [ 1786.918273] Lustre: Skipped 2 previous similar messages [ 1788.826074] Lustre: server umount lustre-MDT0000 complete [ 1792.639466] LustreError: 68409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784585532 with bad export cookie 5531395023205474976 [ 1792.645252] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1792.658149] LustreError: 68409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1793.116606] Lustre: server umount lustre-MDT0001 complete [ 1802.141663] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:12:20 (1784585540) [ 1814.028921] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1853.376318] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 1864.497604] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1902.859285] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1903.125511] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1903.162982] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1903.306984] Lustre: lustre-MDT0000: new disk, initializing [ 1903.388472] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1903.413816] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1908.060957] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1920.142993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1920.354479] Lustre: 76411:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1920.489345] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1920.501132] Lustre: Skipped 1 previous similar message [ 1920.621773] Lustre: lustre-MDT0001: new disk, initializing [ 1920.748873] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1920.781274] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1920.797084] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1926.018505] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1935.659071] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1941.297520] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1947.611164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1947.833476] Lustre: lustre-OST0000: new disk, initializing [ 1947.838960] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1947.846443] Lustre: 78383:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1947.934514] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1954.211340] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1957.489461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1957.505634] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1957.663118] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1963.577268] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1967.619666] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1967.752161] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1971.139977] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1971.338487] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1973.218902] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1973.220269] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1973.223797] LustreError: Skipped 1 previous similar message [ 1973.226545] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1973.254692] Lustre: Skipped 1 previous similar message [ 1973.265701] Lustre: Skipped 1 previous similar message [ 1978.338725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1978.374856] Lustre: Skipped 1 previous similar message [ 1979.062915] Lustre: server umount lustre-OST0000 complete [ 1987.559518] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1987.566319] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1987.589530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1998.303125] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1998.596591] Lustre: server umount lustre-MDT0000 complete [ 1998.820635] LustreError: 77595:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1998.843622] LustreError: 77595:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2002.463640] LustreError: 76404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784585742 with bad export cookie 5531395023205475858 [ 2002.476429] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2002.756266] Lustre: server umount lustre-MDT0001 complete [ 2012.695775] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:15:51 (1784585751) [ 2020.344031] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2021.011022] LustreError: 80797:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2021.136316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2025.789590] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2026.469152] LustreError: 80798:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2030.563322] LustreError: 80797:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2033.407447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2038.461455] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2046.917408] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2052.787744] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2058.659886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2059.049940] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2059.061464] Lustre: Skipped 1 previous similar message [ 2065.207683] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2072.077785] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2075.548579] Lustre: 83468:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting d0d4020d-76e2-4bc7-98ac-be7c9fb27b1c at adminstrative request [ 2076.642558] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2076.656063] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2086.882781] Lustre: MGS: Client d0d4020d-76e2-4bc7-98ac-be7c9fb27b1c (at 0@lo) reconnecting [ 2086.888534] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x4cc376778e86a984 to 0x4cc376778e86ac1d [ 2086.903090] Lustre: MGC192.168.203.135@tcp: Connection restored to 0@lo (at 0@lo) [ 2112.836472] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2113.058872] Lustre: lustre-OST0001: new disk, initializing [ 2113.061800] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2113.067724] Lustre: 83861:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2119.123648] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2122.871414] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2122.888793] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2123.028548] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2128.228991] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2133.474051] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2133.478439] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2133.492976] Lustre: Skipped 1 previous similar message [ 2133.496864] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2133.517497] Lustre: Skipped 5 previous similar messages [ 2145.247174] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2145.385533] Lustre: server umount lustre-OST0001 complete [ 2148.841617] LustreError: 83499:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2148.854946] LustreError: 83499:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2150.369620] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2150.393133] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2150.415018] Lustre: Skipped 1 previous similar message [ 2150.425787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2150.428389] Lustre: Skipped 4 previous similar messages [ 2159.076479] LustreError: 83499:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2159.093263] LustreError: 83499:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2164.191128] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2164.282665] Lustre: server umount lustre-OST0000 complete [ 2169.314349] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2169.326044] Lustre: Skipped 1 previous similar message [ 2174.117214] Lustre: server umount lustre-MDT0000 complete [ 2177.382534] LustreError: 80777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784585917 with bad export cookie 5531395023205477405 [ 2177.388354] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2177.394102] LustreError: 80777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2177.756657] Lustre: server umount lustre-MDT0001 complete [ 2184.768768] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:18:43 (1784585923) [ 2186.164626] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2187.857043] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:18:46 (1784585926) [ 2189.324391] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2191.146899] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:18:50 (1784585930) [ 2192.882456] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2194.544533] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:18:53 (1784585933) [ 2196.230476] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2197.941280] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:18:56 (1784585936) [ 2199.369751] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2205.397878] loop: module loaded [ 2205.404839] loop0: detected capacity change from 0 to 4294967296 [ 2205.459157] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2207.452650] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2213.008631] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2218.926250] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2247.851838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2247.917855] LustreError: 87985:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2247.928285] LustreError: 87985:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2247.945288] LustreError: 87985:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2247.957252] LustreError: 87985:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2268.166446] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 2279.532043] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 2313.883587] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2314.072610] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2314.111144] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2314.181593] Lustre: lustre-MDT0000: new disk, initializing [ 2314.257044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2314.260556] Lustre: Skipped 1 previous similar message [ 2314.276419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2318.258868] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2328.967892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2329.094699] Lustre: 90850:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2329.463320] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2334.565703] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2345.441364] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2351.744296] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2358.138976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2358.313553] Lustre: 92821:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2364.055361] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2365.551438] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2365.723403] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2375.664573] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2382.297822] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2382.612881] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2388.095683] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2388.277992] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2391.532061] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2391.533570] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2391.547012] LustreError: Skipped 2 previous similar messages [ 2391.548234] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2391.548240] Lustre: Skipped 8 previous similar messages [ 2391.584284] Lustre: Skipped 2 previous similar messages [ 2396.465263] Lustre: server umount lustre-OST0000 complete [ 2406.808602] Lustre: server umount lustre-MDT0000 complete [ 2406.886509] LustreError: 90862:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2406.909592] LustreError: 90862:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2410.190456] LustreError: 91657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784586149 with bad export cookie 5531395023205478028 [ 2410.195041] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2410.201362] LustreError: 91657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2410.410618] Lustre: server umount lustre-MDT0001 complete [ 2419.698797] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:22:38 (1784586158) [ 2426.634759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2431.893883] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2440.682057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2445.772317] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2453.001283] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2458.287794] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2465.333453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2465.688972] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2465.699061] Lustre: Skipped 4 previous similar messages [ 2472.162615] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2479.054805] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2483.280441] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2483.504565] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2487.762858] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2487.943940] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2494.501543] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2494.682519] Lustre: lustre-OST0001: new disk, initializing [ 2494.687618] Lustre: Skipped 2 previous similar messages [ 2494.692567] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2494.699575] Lustre: Skipped 3 previous similar messages [ 2494.708969] Lustre: 98604:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2501.294863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2501.310191] Lustre: Skipped 2 previous similar messages [ 2501.320705] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2501.419551] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2501.594228] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2511.286585] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2515.940601] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2516.159189] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2520.809108] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2520.979391] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2539.728977] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2565.214670] Lustre: Failing over lustre-MDT0000 [ 2565.693922] Lustre: server umount lustre-MDT0000 complete [ 2568.167512] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2568.175628] LustreError: 95227:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2568.193630] Lustre: Skipped 5 previous similar messages [ 2568.220747] LustreError: 95227:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 2583.519777] Lustre: 66641:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784586307/real 1784586307] req@000000004fee9f29 x1871273431152000/t0(0) o400->MGC192.168.203.135@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1784586323 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2583.566913] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2590.723870] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2590.726853] LDISKFS-fs (dm-0): recovery complete [ 2590.733105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2593.770516] LustreError: 66640:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000d952c910 x1871273431160192/t0(0) o250->MGC192.168.203.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2594.129889] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2594.630372] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2594.632990] Lustre: 99860:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2599.173657] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2599.414384] Lustre: 95972:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2599.424164] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2599.435913] Lustre: 95972:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 2599.597742] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2599.921541] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2600.269440] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2600.282981] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2600.612370] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2600.624383] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2600.642032] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2600.657259] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 2601.304114] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2601.317566] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2601.647681] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2601.658912] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2602.933610] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2602.946232] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 2603.587795] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2603.602107] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 2603.927126] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2603.938485] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 2607.104051] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2607.113970] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 2607.763087] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2607.783587] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2608.103122] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2608.115911] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2615.407908] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2615.424971] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 2616.072291] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2616.081926] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 2616.407201] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2616.423799] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 2631.612688] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2631.633854] Lustre: 101450:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 2632.273551] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2632.291522] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2632.615140] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2632.621505] LustreError: 101450:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2654.135158] Lustre: 101450:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2654.148255] Lustre: 101450:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2654.163708] Lustre: 101450:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 0e7b6a84-ae72-49cb-8eaa-f654c0624c10@192.168.203.35@tcp [ 2654.193612] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2654.203195] Lustre: 101450:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000827ecea8 x1871273416264192/t0(8589934768) o35->0e7b6a84-ae72-49cb-8eaa-f654c0624c10@192.168.203.35@tcp:454/0 lens 392/0 e 0 to 0 dl 1784586404 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2654.252142] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2654.268982] Lustre: lustre-MDT0000: Denying connection for new client 0e7b6a84-ae72-49cb-8eaa-f654c0624c10 (at 192.168.203.35@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2654.278542] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2654.330686] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2654.333703] Lustre: Skipped 2 previous similar messages [ 2654.338644] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2654.377248] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2654.383189] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2675.698593] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2675.721258] LustreError: Skipped 1 previous similar message [ 2675.734392] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2675.750828] Lustre: Skipped 3 previous similar messages [ 2681.029699] Lustre: server umount lustre-OST0000 complete [ 2681.327132] LustreError: 97055:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2681.359684] LustreError: 97055:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 33 previous similar messages [ 2700.255638] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2700.474757] Lustre: server umount lustre-OST0001 complete [ 2705.888805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2705.911346] Lustre: Skipped 4 previous similar messages [ 2711.467983] Lustre: server umount lustre-MDT0000 complete [ 2715.917451] LustreError: 95211:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784586455 with bad export cookie 5531395023205529240 [ 2715.921079] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2715.924752] LustreError: 95211:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2716.183185] Lustre: server umount lustre-MDT0001 complete [ 2725.338939] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 18:27:44 (1784586464) [ 2733.866176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2733.885939] Lustre: *** cfs_fail_loc=197, val=0*** [ 2749.537300] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2754.655627] LustreError: 103428:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.135@tcp: failed processing log, type 4: rc = -110 [ 2759.826733] Lustre: *** cfs_fail_loc=197, val=0*** [ 2759.840552] Lustre: Skipped 65 previous similar messages [ 2780.383498] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2780.394879] Lustre: Skipped 2 previous similar messages [ 2786.893423] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2794.132950] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2796.089513] Lustre: server umount lustre-OST0000 complete [ 2803.806465] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 18:29:02 (1784586542) [ 2824.578693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2825.028961] LustreError: 105049:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2825.043152] LustreError: 105049:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 2829.750342] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2839.580579] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2843.312768] LustreError: 105080:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2843.335191] LustreError: 105080:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2843.355325] LustreError: 105080:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 2843.926467] Lustre: server umount lustre-MDT0000 complete [ 2877.950241] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 2889.544680] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 2928.850033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2929.155655] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2929.215678] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2929.361511] Lustre: lustre-MDT0000: new disk, initializing [ 2929.605729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2934.650389] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2944.747517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2944.946653] Lustre: 109254:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2945.198412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2949.169313] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2958.986546] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2963.714567] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2970.639203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2970.926398] Lustre: 111224:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2976.965982] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2977.438143] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2985.864612] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2990.916255] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2991.152503] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2996.115606] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2996.344926] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3002.853192] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3002.864635] LustreError: Skipped 3 previous similar messages [ 3002.872444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3002.897662] Lustre: Skipped 1 previous similar message [ 3002.905782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3002.923953] Lustre: Skipped 13 previous similar messages [ 3004.510742] Lustre: server umount lustre-OST0000 complete [ 3022.304038] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3022.427498] Lustre: server umount lustre-MDT0000 complete [ 3026.088071] LustreError: 109249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784586765 with bad export cookie 5531395023205539817 [ 3026.095193] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3026.100602] LustreError: 109249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3026.372525] Lustre: server umount lustre-MDT0001 complete [ 3035.419323] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 18:32:54 (1784586774) [ 3045.449914] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 3048.416383] Key type lgssc unregistered [ 3048.711166] LNet: 113715:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3048.735888] LNetError: 113715:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3049.765787] LNet: Removed LNI 192.168.203.135@tcp [ 3050.573607] Key type .llcrypt unregistered [ 3050.579475] Key type ._llcrypt unregistered [ 3069.787520] Key type ._llcrypt registered [ 3069.792442] Key type .llcrypt registered [ 3069.890643] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 3084.474446] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3085.330210] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3085.385972] alg: No test for adler32 (adler32-zlib) [ 3086.400354] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 3086.671582] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 3088.359187] Key type lgssc registered [ 3089.546307] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3128.232604] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3128.265912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3129.587596] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3129.644482] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3129.738839] Lustre: lustre-MDT0000: new disk, initializing [ 3129.800554] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3129.823027] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3133.741450] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3144.646652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3145.000813] Lustre: lustre-OST0000: new disk, initializing [ 3145.009183] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3145.019146] Lustre: Skipped 1 previous similar message [ 3145.029584] Lustre: 117891:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3145.162755] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3151.001083] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3151.017687] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3151.075143] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3151.806390] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3161.573637] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3161.871305] Lustre: lustre-OST0001: new disk, initializing [ 3161.877980] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3161.884513] Lustre: 118755:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3161.963439] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3169.573167] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3171.077880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3171.103944] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3171.166628] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3185.463767] Lustre: server umount lustre-MDT0000 complete [ 3194.395478] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3197.945382] LustreError: 117078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784586937 with bad export cookie 12365238207220095023 [ 3197.955449] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3202.911542] Lustre: 115098:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784586926/real 1784586926] req@00000000cbf6a84a x1871274915684608/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784586942 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3202.932433] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3208.095191] Lustre: 115098:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784586931/real 1784586931] req@0000000013cab42c x1871274915684992/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784586947 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3208.327144] Lustre: server umount lustre-OST0000 complete [ 3212.262729] Lustre: 115097:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784586936/real 1784586936] req@00000000744e9fe8 x1871274915685376/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784586952 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3212.531701] Lustre: server umount lustre-OST0001 complete [ 3221.919835] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 18:36:00 (1784586960) [ 3242.699988] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 18:36:21 (1784586981) [ 3253.182743] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3274.259816] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 18:36:53 (1784587013) [ 3313.267601] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 3323.580484] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3361.765773] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3361.949893] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3361.973911] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3362.029682] Lustre: lustre-MDT0000: new disk, initializing [ 3362.092092] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3362.111595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3365.644331] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3376.255498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3376.402986] Lustre: 126508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3376.485327] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3376.488091] Lustre: Skipped 1 previous similar message [ 3376.533719] Lustre: lustre-MDT0001: new disk, initializing [ 3376.601028] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3376.632245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3376.642308] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3380.694119] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3389.548994] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3395.127729] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3401.886074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3402.159612] Lustre: lustre-OST0000: new disk, initializing [ 3402.165675] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3402.178259] Lustre: 128473:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3402.299481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3408.531947] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3410.005690] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3410.011492] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3410.181757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3419.075547] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3424.396917] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3424.659793] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3429.320527] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3429.531558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3436.001897] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3436.003860] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3436.007708] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3436.053364] Lustre: Skipped 1 previous similar message [ 3437.504993] Lustre: server umount lustre-OST0000 complete [ 3441.125215] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3441.136247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3441.140307] Lustre: Skipped 2 previous similar messages [ 3446.241117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3446.250270] Lustre: Skipped 1 previous similar message [ 3447.421297] Lustre: server umount lustre-MDT0000 complete [ 3451.365906] LustreError: 126497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784587191 with bad export cookie 12365238207220096248 [ 3451.389481] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3451.396337] LustreError: 126497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3452.632275] Lustre: server umount lustre-MDT0001 complete [ 3462.005225] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:40:00 (1784587200) [ 3469.767122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3470.317430] LustreError: 130890:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3470.342613] LustreError: 130890:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3470.454663] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3475.193713] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3475.957190] LustreError: 130891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3481.056673] LustreError: 130890:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3483.412046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3488.726978] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3495.849758] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3501.996906] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3508.837420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3509.278458] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3509.287348] Lustre: Skipped 1 previous similar message [ 3516.162362] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3524.403523] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3532.288477] Lustre: *** cfs_fail_loc=159, val=0*** [ 3532.298409] Lustre: Skipped 1 previous similar message [ 3547.981531] Lustre: lustre-MDT0000: Client f26e136d-555c-4686-9f2e-b4a6cbef3cfc (at 192.168.203.35@tcp) reconnecting [ 3550.179235] Lustre: *** cfs_fail_loc=159, val=0*** [ 3550.186434] Lustre: Skipped 4 previous similar messages [ 3565.898340] Lustre: lustre-MDT0000: Client f26e136d-555c-4686-9f2e-b4a6cbef3cfc (at 192.168.203.35@tcp) reconnecting [ 3572.705130] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3572.713546] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3572.724142] Lustre: Skipped 1 previous similar message [ 3572.730037] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3574.757087] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3576.106814] Lustre: server umount lustre-OST0000 complete [ 3579.873942] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3579.886560] Lustre: Skipped 1 previous similar message [ 3579.891739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3579.898123] Lustre: Skipped 3 previous similar messages [ 3585.865810] Lustre: server umount lustre-MDT0000 complete [ 3589.488654] LustreError: 130870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784587329 with bad export cookie 12365238207220097130 [ 3589.494202] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3589.507645] LustreError: 130870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3590.039916] Lustre: server umount lustre-MDT0001 complete [ 3600.086563] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 3602.641749] Key type lgssc unregistered [ 3602.919368] LNet: 134668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3602.925068] LNetError: 134668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3603.942031] LNet: Removed LNI 192.168.203.135@tcp [ 3604.786914] Key type .llcrypt unregistered [ 3604.790683] Key type ._llcrypt unregistered [ 3629.603645] Key type ._llcrypt registered [ 3629.606383] Key type .llcrypt registered [ 3629.700328] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3630.929430] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3630.968082] alg: No test for adler32 (adler32-zlib) [ 3632.048570] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 3632.312536] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 3633.992452] Key type lgssc registered [ 3635.091780] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3641.346720] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3641.381735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3642.808397] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3646.371924] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3653.092463] LustreError: 136083:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3653.106195] LustreError: 136083:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3653.609100] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3653.887903] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3658.089583] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3665.172146] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3670.407025] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3676.525697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3676.802986] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3682.511750] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3688.119835] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3688.289178] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3697.425324] Lustre: *** cfs_fail_loc=159, val=0*** [ 3712.831567] Lustre: lustre-MDT0000: Client 20dd9bd6-9414-44ee-8367-e1bf379c86e6 (at 192.168.203.35@tcp) reconnecting [ 3715.006636] Lustre: *** cfs_fail_loc=159, val=0*** [ 3731.320588] Lustre: lustre-MDT0001: Client 20dd9bd6-9414-44ee-8367-e1bf379c86e6 (at 192.168.203.35@tcp) reconnecting [ 3732.963268] Lustre: *** cfs_fail_loc=159, val=0*** [ 3732.969636] Lustre: Skipped 3 previous similar messages [ 3749.206614] Lustre: lustre-MDT0001: Client 20dd9bd6-9414-44ee-8367-e1bf379c86e6 (at 192.168.203.35@tcp) reconnecting [ 3753.942114] Lustre: 139114:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3756.847293] Lustre: *** cfs_fail_loc=159, val=0*** [ 3756.863201] Lustre: Skipped 5 previous similar messages [ 3773.282341] Lustre: lustre-MDT0000: Client ef24dcf5-73e8-40d9-a69b-9e19efaa5153 (at 192.168.203.35@tcp) reconnecting [ 3774.965068] Lustre: *** cfs_fail_loc=159, val=0*** [ 3774.969814] Lustre: Skipped 12 previous similar messages [ 3791.189250] Lustre: lustre-MDT0000: Client ef24dcf5-73e8-40d9-a69b-9e19efaa5153 (at 192.168.203.35@tcp) reconnecting [ 3792.662660] Lustre: 139406:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3799.015068] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3799.016304] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3799.023546] LustreError: Skipped 2 previous similar messages [ 3799.041143] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3801.058404] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3801.061196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3801.062061] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3801.062067] LustreError: Skipped 3 previous similar messages [ 3801.085924] Lustre: Skipped 1 previous similar message [ 3806.177742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3806.184770] Lustre: Skipped 1 previous similar message [ 3811.296232] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3811.298849] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3811.310720] Lustre: Skipped 1 previous similar message [ 3811.513645] Lustre: server umount lustre-OST0000 complete [ 3816.422257] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3816.441679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3816.448700] Lustre: Skipped 1 previous similar message [ 3826.656618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3826.668162] Lustre: Skipped 3 previous similar messages [ 3829.215128] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3831.428911] Lustre: server umount lustre-MDT0000 complete [ 3831.782876] LustreError: 138592:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3831.793055] LustreError: 138592:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3834.578058] LustreError: 139233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784587574 with bad export cookie 8101922005800868574 [ 3834.585533] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3834.595114] LustreError: 139233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3834.862374] Lustre: server umount lustre-MDT0001 complete [ 3845.468851] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 3847.767628] Key type lgssc unregistered [ 3847.968715] LNet: 140461:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3847.981720] LNetError: 140461:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3848.008219] LNet: Removed LNI 192.168.203.135@tcp [ 3848.773692] Key type .llcrypt unregistered [ 3848.776805] Key type ._llcrypt unregistered [ 3865.551252] Key type ._llcrypt registered [ 3865.553664] Key type .llcrypt registered [ 3865.645934] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3866.460436] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3866.490172] alg: No test for adler32 (adler32-zlib) [ 3867.521588] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 3867.735834] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 3869.399133] Key type lgssc registered [ 3870.631648] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3877.679598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3877.707516] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3879.207161] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3883.190810] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3888.625023] LustreError: 141858:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3888.653393] LustreError: 141858:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3890.691141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3891.147564] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3895.627117] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3902.103573] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3907.872506] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3914.429833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3914.680684] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3915.698981] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3920.051443] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3923.892923] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3927.049465] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3933.974535] Lustre: 144439:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3937.411643] Lustre: 144488:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3938.986185] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3939.841196] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3943.401743] Lustre: 144635:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3944.277884] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3945.130764] Lustre: 144733:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3949.543603] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3949.544751] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3949.547056] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3949.566028] Lustre: Skipped 1 previous similar message [ 3954.659579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3954.667303] Lustre: Skipped 2 previous similar messages [ 3955.047104] Lustre: server umount lustre-OST0000 complete [ 3959.779574] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3959.782142] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3959.807615] Lustre: Skipped 1 previous similar message [ 3964.836474] Lustre: server umount lustre-MDT0000 complete [ 3964.896716] LustreError: 143761:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3964.918590] LustreError: 143761:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3968.283155] LustreError: 141837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784587708 with bad export cookie 17966671315216185395 [ 3968.286340] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3968.303709] LustreError: 141837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3969.030690] Lustre: server umount lustre-MDT0001 complete [ 3981.121472] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 3984.726502] Key type lgssc unregistered [ 3985.031568] LNet: 145795:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3985.037383] LNetError: 145795:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3985.063658] LNet: Removed LNI 192.168.203.135@tcp [ 3986.200294] Key type .llcrypt unregistered [ 3986.202268] Key type ._llcrypt unregistered [ 4010.462497] Key type ._llcrypt registered [ 4010.465192] Key type .llcrypt registered [ 4010.574704] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4011.312450] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4011.335780] alg: No test for adler32 (adler32-zlib) [ 4012.400859] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 4012.616667] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4014.263170] Key type lgssc registered [ 4015.245543] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4022.299854] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4022.328321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4023.852350] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4028.159303] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4034.017076] LustreError: 147209:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4034.052140] LustreError: 147209:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4036.653185] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4037.028528] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4041.110640] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4048.057225] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4053.045324] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4059.140870] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4059.546227] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4061.647763] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4065.671439] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4068.793510] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4073.037943] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4079.369267] Lustre: *** cfs_fail_loc=159, val=0*** [ 4094.775510] Lustre: lustre-MDT0000: Client b08ea52a-a389-4fa5-aa2e-ffa2f4cfcadf (at 192.168.203.35@tcp) reconnecting [ 4099.553539] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4099.555170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4099.555637] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4099.564416] Lustre: Skipped 1 previous similar message [ 4099.577247] Lustre: Skipped 1 previous similar message [ 4104.687162] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4104.696817] Lustre: Skipped 1 previous similar message [ 4105.117124] Lustre: server umount lustre-OST0000 complete [ 4108.768446] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4108.775662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4108.797425] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4109.803818] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4114.867177] Lustre: server umount lustre-MDT0000 complete [ 4114.915992] LustreError: 147209:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4114.925731] LustreError: 147209:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4118.894504] LustreError: 147189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784587858 with bad export cookie 5343899002925128990 [ 4118.912494] LustreError: 147189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4118.914776] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4119.298514] Lustre: server umount lustre-MDT0001 complete [ 4133.581390] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4137.076724] Key type lgssc unregistered [ 4137.510537] LNet: 150871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4137.522646] LNetError: 150871:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4137.533852] LNet: Removed LNI 192.168.203.135@tcp [ 4138.716112] Key type .llcrypt unregistered [ 4138.719533] Key type ._llcrypt unregistered [ 4162.609450] Key type ._llcrypt registered [ 4162.611901] Key type .llcrypt registered [ 4162.711520] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4163.331901] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4163.348785] alg: No test for adler32 (adler32-zlib) [ 4164.367547] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 4164.709313] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4166.439534] Key type lgssc registered [ 4168.052206] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4176.655062] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4176.666653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4178.097426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4181.959636] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4183.523945] LustreError: 152286:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4183.545294] LustreError: 152286:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4187.625509] LustreError: 152287:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4190.006506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4190.478200] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4195.352095] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4204.524512] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4210.672065] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4217.761148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4218.336187] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4222.486579] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4226.333513] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4233.681766] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4235.774836] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4253.013241] Lustre: lustre-OST0000: evicting 7f945f69-9c63-42a1-a33d-528f0b67fb1a (at 192.168.203.35@tcp) by administrative request [ 4253.020181] Lustre: Skipped 1 previous similar message [ 4283.875707] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4283.887106] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4283.908537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4286.946437] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4286.946583] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4286.952986] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4292.070718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4292.086870] Lustre: Skipped 2 previous similar messages [ 4296.159565] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4296.400289] Lustre: server umount lustre-OST0000 complete [ 4302.308871] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4302.311526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4302.329377] Lustre: Skipped 1 previous similar message [ 4302.347585] Lustre: Skipped 1 previous similar message [ 4306.015482] Lustre: server umount lustre-MDT0000 complete [ 4307.435327] LustreError: 152281:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4307.445583] LustreError: 152281:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4309.575640] LustreError: 153030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784588049 with bad export cookie 1766053481756295435 [ 4309.582464] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4309.589434] LustreError: 153030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4309.954689] Lustre: server umount lustre-MDT0001 complete [ 4322.377926] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4324.991854] Key type lgssc unregistered [ 4325.389782] LNet: 156368:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4325.410096] LNetError: 156368:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4325.436267] LNet: Removed LNI 192.168.203.135@tcp [ 4326.505872] Key type .llcrypt unregistered [ 4326.513225] Key type ._llcrypt unregistered [ 4398.063455] Key type ._llcrypt registered [ 4398.065263] Key type .llcrypt registered [ 4398.138440] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4398.675549] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4398.754144] alg: No test for adler32 (adler32-zlib) [ 4399.743886] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 4399.884770] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4401.591412] Key type lgssc registered [ 4402.643634] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4409.968176] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4409.980109] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4411.380231] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4415.062099] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4421.601766] LustreError: 158858:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4421.618816] LustreError: 158858:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4421.694157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4422.038240] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4425.915644] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4432.684244] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4437.095754] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4443.013577] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4443.349338] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4446.443389] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4449.286839] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4456.504456] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4456.687335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4467.170947] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4467.177740] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4467.178114] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4467.202484] Lustre: Skipped 1 previous similar message [ 4470.101372] Lustre: server umount lustre-OST0000 complete [ 4477.408588] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4477.412027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4477.424155] Lustre: Skipped 2 previous similar messages [ 4479.774595] Lustre: server umount lustre-MDT0000 complete [ 4482.538667] LustreError: 160590:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4482.564347] LustreError: 160590:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4483.488094] LustreError: 158838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784588223 with bad export cookie 11669678998675547520 [ 4483.489392] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4483.504800] LustreError: 158838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4483.871397] Lustre: server umount lustre-MDT0001 complete [ 4495.050186] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4497.935686] Key type lgssc unregistered [ 4498.177731] LNet: 162400:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4498.185410] LNetError: 162400:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4499.239786] LNet: Removed LNI 192.168.203.135@tcp [ 4500.093472] Key type .llcrypt unregistered [ 4500.095228] Key type ._llcrypt unregistered [ 4522.252686] Key type ._llcrypt registered [ 4522.255669] Key type .llcrypt registered [ 4522.351453] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4523.210727] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4523.302491] alg: No test for adler32 (adler32-zlib) [ 4524.328665] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 4524.581717] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4526.279157] Key type lgssc registered [ 4527.522312] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4541.044904] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 18:58:00 (1784588280) [ 4555.407607] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 4566.122198] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4597.454232] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4597.480570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4598.706285] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4598.743867] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4598.816327] Lustre: lustre-MDT0000: new disk, initializing [ 4598.935062] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4598.952115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4602.545100] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4610.993041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4611.099405] Lustre: 166664:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4611.133298] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4611.136427] Lustre: Skipped 1 previous similar message [ 4611.200053] Lustre: lustre-MDT0001: new disk, initializing [ 4611.308890] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4611.356381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4611.373347] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4614.635114] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4622.444194] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4627.261735] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4632.531757] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4632.803819] Lustre: lustre-OST0000: new disk, initializing [ 4632.807171] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4632.813170] Lustre: 168629:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4632.903791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4638.629069] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4641.363231] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4641.378093] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4641.521795] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4646.826179] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4657.121416] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4657.133989] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4657.146233] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4657.157313] Lustre: Skipped 1 previous similar message [ 4657.173541] Lustre: Skipped 1 previous similar message [ 4661.499603] Lustre: server umount lustre-OST0000 complete [ 4667.367964] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4667.382062] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4672.454290] Lustre: server umount lustre-MDT0000 complete [ 4675.510202] LustreError: 168628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784588415 with bad export cookie 7469697915819610033 [ 4675.517267] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4675.524823] LustreError: 168628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4675.831829] Lustre: server umount lustre-MDT0001 complete [ 4685.057350] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4687.793113] Key type lgssc unregistered [ 4688.084458] LNet: 170531:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4688.093932] LNetError: 170531:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4688.108215] LNet: Removed LNI 192.168.203.135@tcp [ 4688.873150] Key type .llcrypt unregistered [ 4688.876834] Key type ._llcrypt unregistered [ 4710.543228] Key type ._llcrypt registered [ 4710.548057] Key type .llcrypt registered [ 4710.664617] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4711.644554] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4711.691360] alg: No test for adler32 (adler32-zlib) [ 4712.739743] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 4712.975913] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4714.631162] Key type lgssc registered [ 4715.746916] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4723.289234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4723.339805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4725.073829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4730.297297] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4734.461430] LustreError: 171959:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4734.489840] LustreError: 171959:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4739.570569] LustreError: 171958:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4741.317230] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4741.720931] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4746.645695] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4752.312805] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4756.705906] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4762.941124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4763.257412] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4769.252297] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4769.419660] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4775.870949] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4780.912978] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4780.936064] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4780.952225] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4780.958188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.975036] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4781.879639] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4781.911586] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4781.937650] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4781.944655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.957457] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4782.696892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.401826] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4783.409430] Lustre: Skipped 1 previous similar message [ 4783.420280] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4783.428881] Lustre: Skipped 1 previous similar message [ 4783.433813] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4783.437103] Lustre: Skipped 1 previous similar message [ 4783.445849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.456587] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4783.461651] Lustre: Skipped 1 previous similar message [ 4784.080189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.648438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.264503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.946033] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4785.949269] Lustre: Skipped 3 previous similar messages [ 4785.964468] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4785.977423] Lustre: Skipped 3 previous similar messages [ 4785.980911] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4785.985097] Lustre: Skipped 3 previous similar messages [ 4785.987348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.995658] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4785.999947] Lustre: Skipped 3 previous similar messages [ 4786.780330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.445775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.427791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.213336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.897528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.621381] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4790.632312] Lustre: Skipped 5 previous similar messages [ 4790.650512] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4790.663901] Lustre: Skipped 5 previous similar messages [ 4790.669816] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4790.676948] Lustre: Skipped 5 previous similar messages [ 4790.680840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.691598] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4790.702031] Lustre: Skipped 5 previous similar messages [ 4791.611882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.583750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.663286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.830251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.141209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.234704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.509481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.608225] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4799.621732] Lustre: Skipped 7 previous similar messages [ 4799.646802] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4799.667987] Lustre: Skipped 7 previous similar messages [ 4799.684598] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4799.687415] Lustre: Skipped 7 previous similar messages [ 4799.720772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.738052] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4799.752872] Lustre: Skipped 7 previous similar messages [ 4800.831965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.118138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.933396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.482252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.796129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.027294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.459026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.604702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.685096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.735561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.608621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.581844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.375820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.117652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.784953] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4815.789496] Lustre: Skipped 14 previous similar messages [ 4815.804214] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4815.826505] Lustre: Skipped 14 previous similar messages [ 4815.830747] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4815.837340] Lustre: Skipped 14 previous similar messages [ 4815.841902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.860681] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4815.875161] Lustre: Skipped 14 previous similar messages [ 4816.479109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.350148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.213234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.074577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.053362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.840973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.572899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.406173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.247020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.102871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.868848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.735394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.613105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.253821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.917754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.792277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.711542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.381469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.050062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.732158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.691412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.392706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.184874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.964250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.638887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.423751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.200502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.969658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.891548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.750877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.508750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.211371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.967402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.782784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.700742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.560646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.464802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.345630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.226277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.145466] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4848.151714] Lustre: Skipped 39 previous similar messages [ 4848.164466] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4848.182551] Lustre: Skipped 39 previous similar messages [ 4848.191224] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4848.200780] Lustre: Skipped 39 previous similar messages [ 4848.206235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.218073] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4848.229388] Lustre: Skipped 39 previous similar messages [ 4849.095917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.901070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.753506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.548307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.330462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.123135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.942064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.702250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.433317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.246162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.058248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.838633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.648721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.522658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.396647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.183486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.952823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.915726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.811193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.585429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.485773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.296911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.038832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.880042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.616438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.332795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.081795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.873589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.648146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.337211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.158483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.877443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.589361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.335365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.979235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.627279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.423815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.241552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.045933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.750675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.437662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.174746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.904538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.612055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.237660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.849064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.471405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.119293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.798759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.413104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.050493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.715496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.380334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.066481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.820098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.487046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.315962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.980314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.679779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.349568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.997072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.608916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.451055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.099038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.922689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.734571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.545334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.314529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.208758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.064950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.891520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.791730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.812566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.613119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.478868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.305496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.176492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.934222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.756913] LustreError: 171978:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4908.768528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.769988] LustreError: 171978:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4909.639565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.480664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.101954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.798718] LustreError: 171978:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4911.808623] LustreError: 171978:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4911.816769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.556496] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4912.561896] Lustre: Skipped 83 previous similar messages [ 4912.574655] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4912.589894] Lustre: Skipped 83 previous similar messages [ 4912.602761] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4912.611619] Lustre: Skipped 83 previous similar messages [ 4912.616825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.638200] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4912.647551] Lustre: Skipped 83 previous similar messages [ 4913.391693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.404740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.296311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.094219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.951823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.701277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.694780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.855514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.946814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.275084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.462892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.837306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.982419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.095826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.048445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.153673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.272092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.260041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.256865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.139369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.008806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.904452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.875989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.837253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.707749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.427558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.256583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.932703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.759636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.402192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.150239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.931375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.611732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.200813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.836024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.456513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.070172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.736035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.451897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.206256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.828803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.469523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.201125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.844849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.650881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.385561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.217186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.017479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.690580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.524168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.357785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.107553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.857729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.656101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.484552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.359805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.319787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.086746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.898431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.521458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.356235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.117257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.785293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.443467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.074691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.761060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.532208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.336757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.069995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.801583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.566078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.312044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.911917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.722978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.428583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.147218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.850325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.603265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.414638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.108194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.930125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.628244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.314842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.045139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.837664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.411780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.181497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.853267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.634843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.510622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.279922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.035828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.986419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.052737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.154644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.343258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.226337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.282121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.362691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.273535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.100951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.979409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.959371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.759993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.566949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.423597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.443706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.256501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.292753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.110528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.871644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.523706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.366166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.177777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.109395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.875472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.610123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.232471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.981091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.739399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.580866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.428229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.387913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.143274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.997660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.739576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.510193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.242791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.937725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.692455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.382463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.209914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.071473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.873030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.494167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.269435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.934363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.669361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.489229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.274284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.128829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.893270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.665552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.599699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.437947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.380115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.219604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.067207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.831963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.753498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.532042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.539815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.374475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.170195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.018294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.924840] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5040.947343] Lustre: Skipped 155 previous similar messages [ 5040.960890] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5040.981069] Lustre: Skipped 155 previous similar messages [ 5040.988026] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5040.993497] Lustre: Skipped 155 previous similar messages [ 5040.995853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.016476] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5041.025154] Lustre: Skipped 155 previous similar messages [ 5041.752940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.597077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.408763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.331527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.130899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.997100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.762144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.602893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.342782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.128859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.957366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.786040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.611807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.382918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.088910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.720535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.447797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.032245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.678383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.413755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.184169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.920224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.499595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.203207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.930655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.631098] LustreError: 171978:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 5060.636657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.638849] LustreError: 171978:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 5061.370145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.094683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.824563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.572579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.258415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.987120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.757894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.547058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.363581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.136224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.925246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.739672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.462980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.144441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.851421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.655071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.435794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.125043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.825269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.612531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.437556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.272151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.015114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.847314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.496911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.285815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.879948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.500885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.147485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.756939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.437606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.025184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.693754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.460690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.147936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.965602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.704920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.464222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.192843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.963697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.709673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.393624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.037340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.584577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.097963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.631961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.269373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.886077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.730028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.960547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.747951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.713115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.406962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.373968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.379925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.317669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.268641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.247733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.293394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.501481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.439363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.260247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.109457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.027883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.893486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.500855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.216212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.061528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.945030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.730178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.574978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.326272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.222742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.022653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.876274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.713232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.392239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.301103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.075874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.908219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.641755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.412816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.240116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.920258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.624836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.302150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.169480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.096664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.904045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.637325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.346581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.116833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.011958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.762781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.641543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.490107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.291584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.989399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.927165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.787711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.591469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.321972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.162760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.940289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.975629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.811656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.777456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.654094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.509371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.490618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.564901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.541119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.486074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.279732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.358616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.149353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.875534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.746947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.550444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.182477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.915769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.597040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.519586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.479247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.562796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.336944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.131139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.189034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.279351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.107859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.828677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.420298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.121520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.891670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.636850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.407247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.108148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.731152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.446700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.043878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.618928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.226488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.887213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.436975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.190317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.836522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.521753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.153087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.733861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.320873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.043794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.740330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.653553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.523047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.351862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.136761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.046799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.921784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.943228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.881948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.983869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.002758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.015218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.243461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.134501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.045541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.834906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.026769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.829665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.875556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.677664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.553555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.397816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.368750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.177611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.078718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.902468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.858904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.762843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.748590] LustreError: 171978:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5205.756155] LustreError: 171978:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5205.758901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.498200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.377409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.215148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.282705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.965777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.793310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.555530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.444640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.230967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.053714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.744502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.537442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.401469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.197219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.918338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.679806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.447227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.210953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.863857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.569535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.401847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.073412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.755862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.460280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.412556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.208221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.992894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.779301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.506460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.326660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.218783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.962557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.696247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.354544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.278282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.977877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.899205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.944564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.952140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.051265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.061343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.196284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.182772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.338505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.545793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.241304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.269774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.227588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.044644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.068129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.048500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.131930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.956066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.762645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.709651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.632652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.734397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.948288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.708249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.694767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.567252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.457753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.328654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.391779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.201307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.007134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.788564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.548491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.282141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.161959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.067473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.078184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.929978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.711425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.519528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.368174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.265733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.155081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.897761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.508233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.393252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.298185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.240044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.160182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.344093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.389615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.429750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.611713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.853623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.049830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.226475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.336883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.434938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.562968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.681165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.800141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.162900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.392594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.487286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.412452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.163965] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5297.169496] Lustre: Skipped 306 previous similar messages [ 5297.182608] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5297.192404] Lustre: Skipped 306 previous similar messages [ 5297.196305] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5297.200597] Lustre: Skipped 306 previous similar messages [ 5297.205112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.217705] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5297.227592] Lustre: Skipped 306 previous similar messages [ 5297.962251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.187371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.168676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.454126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.489412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.504288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.427244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.447829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.378784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.377728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.257436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.095613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.033200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.902378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.547368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.369297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.402662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.475059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.333074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.322775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.171961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.103053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.978480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.963175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.920035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.803727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.642301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.482735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.357764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.332988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.111395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.966520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.750233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.675378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.417726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.326967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.056910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.951868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.677765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.390388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.054143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.745572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.621510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.617054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.416753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.300406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.139414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.015880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.025307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.028659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.865413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.836774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.599578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.454298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.244072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.075277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.935417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.827421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.704144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.628502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.642981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.559394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.533408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.486207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.280505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.316858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.235460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.066190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.000650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.808772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.928333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.139499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.273685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.409914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.491462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.651509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.627079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.853212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.568204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.731069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.979242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.450504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.498899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.720399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.634180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.608552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.518116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.455645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.324067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.128338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.201750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.036285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.950100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.860846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.755618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.671492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.620435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.509685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.437912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.201522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.957446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.753865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.666733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.656702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.484912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.366103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.327826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.278267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.216924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.262127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.226944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.188776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.114764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.040474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.940278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.986298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.910920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.909543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.648823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.537030] LustreError: 171978:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5408.549507] LustreError: 171978:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5408.570751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.394658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.272915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.995161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.751477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.587513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.469786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.521976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.289206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.205136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.024239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.813962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.614114] LustreError: 171978:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5418.627052] LustreError: 171978:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5418.634271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.606319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.623848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.412602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.296089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.255959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.088394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.013428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.062698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.967244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.992459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.728590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.622704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.545240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.392731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.372154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.321058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.203140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.223684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.065182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.926687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.760981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.731961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.590607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.554291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.400855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.218654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.043407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.008199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.857055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.836487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.761425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.585050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.473601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.344588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.185367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.051781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.816405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.578885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.479725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.572494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.547200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.482535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.451322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.425235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.324051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.364730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.229748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.078623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.998212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.862918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.729492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.769294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.838795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.842207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.591366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.600217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.502687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.315782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.124970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.944936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.860764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.675422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.557954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.303190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.304604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.191418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.991228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.850463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.780445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.596719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.458757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.467915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.483627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.386543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.294993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.250758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.198950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.089506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.005427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.027676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.945603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.930233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.834537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.563635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.169627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.024907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.889788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.809042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.818627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.764713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.659693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.626127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.467158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.318389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.279558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.146626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.985895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.778855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.678558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.508313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.325691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.240076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.120489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.213581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.301483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.245395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.218330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.200663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.180383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.211852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.208574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.071883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.998676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.923777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.736765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.556784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.499902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.337405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.248259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.035564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.953803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.804911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.727691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.619622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.541543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.575143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.573786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.472939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.368805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.472965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.434361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.322915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.257972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.307879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.355113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.539252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.816398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.086974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.309644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.520600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.824353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.145884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.343522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.286356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.434441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.493740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.667754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.813903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.672621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.681296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.367023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.123536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.770520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.480233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.299984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.312287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.156392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.074638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.879630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.698351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.601639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.488256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.378653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.211385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.110641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.058775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.095442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.109716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.982860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.788632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.622076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.540589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.412609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.310634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.132237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.125200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.098873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.153558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.132408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.027582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.034497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.873817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.670497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.722601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.695573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.541840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.290345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.143289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.940128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.906402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.972688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.874916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.881742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.677790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.444081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.275179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.122979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.824534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.722837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.627267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.495638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.319400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.126153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.905626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.728421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.572877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.324666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.120254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.919757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.753696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.450085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.189039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.112887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.890907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.723844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.549279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.378606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.259799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.162325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.930838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.685595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.331148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.302746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.170562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.108368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.995579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.896557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.751467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.566367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.351264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.239751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.214912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.992989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.740515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.582519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.477275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.233566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.122306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.968070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.829752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.651491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.549770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.360905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.221266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.637945] LustreError: 173780:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5996.653673] LustreError: 171978:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6640.612931] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6640.616954] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6640.625189] Lustre: Skipped 376 previous similar messages [ 6640.628030] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6640.635839] Lustre: Skipped 1 previous similar message [ 6644.430966] Lustre: server umount lustre-OST0000 complete [ 6650.850754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6650.859990] Lustre: Skipped 1 previous similar message [ 6655.968415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6655.980229] Lustre: Skipped 1 previous similar message [ 6661.088666] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6661.103749] Lustre: Skipped 1 previous similar message [ 6663.136437] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6663.466694] Lustre: server umount lustre-MDT0000 complete [ 6666.216448] LustreError: 174146:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6666.234887] LustreError: 174146:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6666.872834] LustreError: 171940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784590406 with bad export cookie 12009284840829039790 [ 6666.875578] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6666.882423] LustreError: 171940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6667.260988] Lustre: server umount lustre-MDT0001 complete [ 6677.677436] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 6680.501931] Key type lgssc unregistered [ 6680.785518] LNet: 227190:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6680.797021] LNetError: 227190:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6680.820706] LNet: Removed LNI 192.168.203.135@tcp [ 6681.859696] Key type .llcrypt unregistered [ 6681.861651] Key type ._llcrypt unregistered [ 6706.300386] Key type ._llcrypt registered [ 6706.302564] Key type .llcrypt registered [ 6706.407537] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 6707.245768] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6707.380627] alg: No test for adler32 (adler32-zlib) [ 6708.555081] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 6708.847241] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 6710.583165] Key type lgssc registered [ 6711.884795] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6720.074695] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6720.098212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6721.591477] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6726.973642] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 6731.748973] LustreError: 228606:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6731.769128] LustreError: 228606:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6735.678087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6736.025884] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6740.870861] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 6749.053930] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6756.040363] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6764.179483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6764.517590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6766.616176] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6771.166080] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 6778.651723] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6786.373826] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6788.301381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6788.312972] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6788.318527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.329259] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6788.329722] LustreError: 228625:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6790.445386] Lustre: lustre-OST0000: Not available for connect from 192.168.203.35@tcp (stopping) [ 6791.138324] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6791.145177] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6791.150315] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6791.184601] Lustre: Skipped 1 previous similar message [ 6795.523156] Lustre: lustre-OST0000: Not available for connect from 192.168.203.35@tcp (stopping) [ 6795.541521] Lustre: Skipped 1 previous similar message [ 6796.273201] Lustre: server umount lustre-OST0000 complete [ 6808.407074] LustreError: 228625:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6830.158620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6830.587182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6831.605297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.619944] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.621387] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6837.326939] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 6845.737029] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6847.643385] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6853.155893] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8d5f046a2800.ost_server_uuid 50 [ 6854.735794] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8d5f046a2800.ost_server_uuid in FULL state after 0 sec [ 6860.256240] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6860.267201] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6860.282146] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6862.308354] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6862.326081] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6864.902570] Lustre: server umount lustre-OST0000 complete [ 6868.963122] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6868.972444] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6868.994826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6869.001891] Lustre: Skipped 2 previous similar messages [ 6874.695553] Lustre: server umount lustre-MDT0000 complete [ 6877.665029] LustreError: 229615:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6877.689336] LustreError: 229615:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6878.087357] LustreError: 228585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784590617 with bad export cookie 1228537324901939707 [ 6878.088211] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6878.103653] LustreError: 228585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6878.453380] Lustre: server umount lustre-MDT0001 complete [ 6888.402145] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 6891.627664] Key type lgssc unregistered [ 6891.932813] LNet: 233813:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6891.938061] LNetError: 233813:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6891.952712] LNet: Removed LNI 192.168.203.135@tcp [ 6892.805716] Key type .llcrypt unregistered [ 6892.808111] Key type ._llcrypt unregistered [ 6924.524427] Key type ._llcrypt registered [ 6924.529207] Key type .llcrypt registered [ 6924.667663] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 6925.875604] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6925.945630] alg: No test for adler32 (adler32-zlib) [ 6927.109491] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 6927.407444] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 6929.087347] Key type lgssc registered [ 6930.176233] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6939.461577] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6939.484543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6940.895822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6945.153664] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 6950.371245] LustreError: 235545:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6950.393670] LustreError: 235545:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6954.070784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6954.556501] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6959.497934] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 6962.632859] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6969.833537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6970.248696] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6977.114205] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 6985.607524] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6986.284041] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6986.549986] Lustre: lustre-OST0001: new disk, initializing [ 6986.553686] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6986.560480] Lustre: 237854:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6986.685857] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6992.538100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6992.554416] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6992.780674] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6993.486795] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7004.078534] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7010.367588] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7021.717104] Lustre: server umount lustre-MDT0000 complete [ 7023.596973] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7023.604818] LustreError: 235545:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7023.606918] Lustre: Skipped 3 previous similar messages [ 7023.636372] LustreError: 235545:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7025.970314] LustreError: 235524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784590765 with bad export cookie 15108863425197775133 [ 7025.998490] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7026.680823] Lustre: server umount lustre-MDT0001 complete [ 7041.232063] Lustre: server umount lustre-OST0000 complete [ 7055.302973] Lustre: server umount lustre-OST0001 complete [ 7077.584209] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7086.698894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7087.045632] LustreError: 240958:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7087.120807] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7091.241972] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7092.202433] LustreError: 240959:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7096.296546] LustreError: 240958:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7098.968203] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7099.327678] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7103.037794] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7105.721481] Lustre: 242100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7112.039311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7118.214209] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7124.596497] LustreError: 242454:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7126.073878] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7126.330377] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7126.335550] Lustre: Skipped 1 previous similar message [ 7131.472236] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7132.031595] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7138.520534] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7141.754299] Lustre: 243969:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7152.100299] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7152.103547] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7152.109369] Lustre: Skipped 3 previous similar messages [ 7152.128715] Lustre: Skipped 3 previous similar messages [ 7157.205152] Lustre: server umount mylustre-MDT0000 complete [ 7157.225949] LustreError: 240959:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7157.258059] LustreError: 240959:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7160.545249] LustreError: 241701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784590900 with bad export cookie 15108863425197777884 [ 7160.546775] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7160.560741] LustreError: 241701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7161.008616] Lustre: server umount mylustre-MDT0001 complete [ 7175.278323] Lustre: server umount mylustre-OST0000 complete [ 7178.207132] Lustre: 234809:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784590902/real 1784590902] req@ffff95905ff92680 x1871278943384704/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784590918 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7178.244159] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7179.408519] Lustre: server umount mylustre-OST0001 complete [ 7201.782624] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7212.135687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7212.676969] LustreError: 246278:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7212.803873] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7217.503440] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7225.538517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7230.128541] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7232.832416] Lustre: 247418:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7238.982810] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7245.035952] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7252.812131] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7258.425569] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7259.134442] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 7259.159787] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7265.359451] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7268.364617] Lustre: 249291:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7277.025141] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7277.032774] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7277.044162] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7279.596666] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7279.602807] Lustre: Skipped 3 previous similar messages [ 7281.216794] Lustre: server umount tfs-MDT0000 complete [ 7284.403073] LustreError: 249293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784591024 with bad export cookie 15108863425197779550 [ 7284.409239] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7284.418580] LustreError: 249293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7284.662354] Lustre: server umount tfs-MDT0001 complete [ 7298.319883] Lustre: server umount tfs-OST0000 complete [ 7300.256611] Lustre: 234806:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784591024/real 1784591024] req@ffff959043267480 x1871278943435392/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784591040 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7300.286401] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7300.298174] Lustre: Skipped 3 previous similar messages [ 7301.962908] Lustre: server umount tfs-OST0001 complete [ 7322.908277] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7332.838609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7333.192218] LustreError: 251540:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7333.208443] LustreError: 251540:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7333.268116] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7333.275604] Lustre: Skipped 3 previous similar messages [ 7337.104839] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7344.682861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7349.801162] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7352.362408] Lustre: 252682:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7358.529303] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7364.934823] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7372.586787] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7378.938571] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 7378.938626] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7379.173959] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7386.334457] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7389.941426] Lustre: 254553:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7396.036536] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:45:35 (1784591135) [ 7414.755484] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7414.770831] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7414.779401] Lustre: Skipped 3 previous similar messages [ 7414.792628] Lustre: Skipped 3 previous similar messages [ 7418.434423] Lustre: server umount lustre-MDT0000 complete [ 7421.864845] LustreError: 251515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784591161 with bad export cookie 15108863425197781216 [ 7421.867171] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7421.873263] LustreError: 251515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7424.994912] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7436.255171] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7436.501371] Lustre: server umount lustre-MDT0001 complete [ 7454.178094] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7454.318287] Lustre: server umount lustre-OST0000 complete [ 7463.757790] Lustre: server umount lustre-OST0001 complete [ 7469.759625] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 7478.243436] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7507.451661] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7507.451661] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7507.451661] [ 7507.506719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7507.668904] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7507.713570] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7507.820500] Lustre: lustre-MDT0000: new disk, initializing [ 7507.896025] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7507.902308] Lustre: Skipped 3 previous similar messages [ 7507.922779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7511.869930] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7520.029549] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7520.029549] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7520.029549] [ 7520.054184] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7520.163325] Lustre: 257736:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7520.233049] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7520.242279] Lustre: Skipped 1 previous similar message [ 7520.292819] Lustre: lustre-MDT0001: new disk, initializing [ 7520.401860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7520.415139] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7523.666504] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7531.723537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7531.918216] Lustre: lustre-OST0000: new disk, initializing [ 7531.929238] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7531.937554] Lustre: 259318:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7533.928594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7533.941139] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7534.025612] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7536.750807] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7544.822659] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7544.969292] Lustre: lustre-OST0001: new disk, initializing [ 7544.975923] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7544.981301] Lustre: 260190:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7546.887629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7546.895507] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7547.004209] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7549.792856] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7559.331894] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 19:48:18 (1784591298) [ 7571.809828] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 19:48:30 (1784591310) [ 7587.810095] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7587.820470] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7587.828914] Lustre: Skipped 3 previous similar messages [ 7587.839700] Lustre: Skipped 6 previous similar messages [ 7592.135681] Lustre: server umount lustre-OST0000 complete [ 7592.934619] LustreError: 259304:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7592.950690] LustreError: 259304:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 7597.025369] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7601.487065] Lustre: server umount lustre-MDT0000 complete [ 7604.503631] LustreError: 257729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784591344 with bad export cookie 15108863425197782770 [ 7604.510130] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7604.513167] LustreError: 257729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7608.292531] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7608.296862] Lustre: Skipped 4 previous similar messages [ 7610.798926] Lustre: server umount lustre-MDT0001 complete [ 7619.127405] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 7642.079185] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7642.199647] Lustre: server umount lustre-OST0001 complete [ 7647.789445] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 7659.261181] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7684.831763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7685.055755] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7685.078541] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7685.133692] Lustre: lustre-MDT0000: new disk, initializing [ 7685.202205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7688.468256] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7696.915656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7697.016335] Lustre: 264724:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7697.307468] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7700.905842] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7708.706561] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7713.639597] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7719.664905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7719.954048] Lustre: lustre-OST0000: new disk, initializing [ 7719.958067] Lustre: Skipped 1 previous similar message [ 7719.966416] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7719.976161] Lustre: Skipped 2 previous similar messages [ 7719.988810] Lustre: 266693:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7721.653478] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7721.665697] Lustre: Skipped 1 previous similar message [ 7721.676154] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7721.852041] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7725.893738] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7733.652445] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7742.436735] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7742.437573] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7742.446290] LustreError: Skipped 1 previous similar message [ 7742.461856] Lustre: Skipped 4 previous similar messages [ 7742.464618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7742.472514] Lustre: Skipped 2 previous similar messages [ 7745.158897] Lustre: server umount lustre-OST0000 complete [ 7748.576665] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7762.399128] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7762.601265] Lustre: server umount lustre-MDT0000 complete [ 7765.660602] LustreError: 264718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784591505 with bad export cookie 15108863425197784240 [ 7765.666341] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7765.672495] LustreError: 264718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7766.009213] Lustre: server umount lustre-MDT0001 complete [ 7775.484140] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 7777.540734] Key type lgssc unregistered [ 7777.761290] LNet: 268452:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7777.766508] LNetError: 268452:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7777.780992] LNet: Removed LNI 192.168.203.135@tcp [ 7778.576156] Key type .llcrypt unregistered [ 7778.578075] Key type ._llcrypt unregistered [ 7800.231517] Key type ._llcrypt registered [ 7800.233973] Key type .llcrypt registered [ 7800.316593] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7801.025033] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7801.064310] alg: No test for adler32 (adler32-zlib) [ 7802.046459] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 7802.204828] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 7803.847382] Key type lgssc registered [ 7804.772904] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7810.861577] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7810.880709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7812.322235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7815.700186] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7821.943652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7822.223645] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7825.586233] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7831.549741] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7835.879488] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7840.164985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7840.428573] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7844.834819] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7849.783827] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7850.406807] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7852.004564] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7852.009077] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7852.016462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7853.699810] Lustre: server umount lustre-OST0000 complete [ 7857.124835] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7857.128458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7857.136190] Lustre: Skipped 1 previous similar message [ 7862.059949] Lustre: server umount lustre-MDT0000 complete [ 7862.242886] LustreError: 269910:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7862.253022] LustreError: 269910:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7864.561978] LustreError: 272440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784591604 with bad export cookie 8986686195355879020 [ 7864.565909] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7864.569016] LustreError: 272440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7864.765232] Lustre: server umount lustre-MDT0001 complete [ 7871.530677] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 7873.375447] Key type lgssc unregistered [ 7873.594542] LNet: 273495:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7873.601605] LNetError: 273495:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7873.616696] LNet: Removed LNI 192.168.203.135@tcp [ 7874.194481] Key type .llcrypt unregistered [ 7874.195968] Key type ._llcrypt unregistered [ 7884.993274] Key type ._llcrypt registered [ 7884.995963] Key type .llcrypt registered [ 7885.059914] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7885.565411] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7885.644462] alg: No test for adler32 (adler32-zlib) [ 7886.586206] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 7886.757728] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 7888.383219] Key type lgssc registered [ 7889.127890] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7894.931536] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7894.950671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7910.496383] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7915.615239] LustreError: 274670:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.135@tcp: failed processing log, type 4: rc = -110 [ 7942.239206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7946.538220] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7951.738687] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7953.287403] Lustre: server umount lustre-OST0000 complete [ 7964.413847] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 7966.051558] Key type lgssc unregistered [ 7966.263223] LNet: 276299:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7966.271718] LNetError: 276299:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7966.286631] LNet: Removed LNI 192.168.203.135@tcp [ 7966.863659] Key type .llcrypt unregistered [ 7966.865364] Key type ._llcrypt unregistered [ 7990.156171] Key type ._llcrypt registered [ 7990.157993] Key type .llcrypt registered [ 7990.229444] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7990.748184] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7990.767093] alg: No test for adler32 (adler32-zlib) [ 7991.736210] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 7991.920107] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 7993.575161] Key type lgssc registered [ 7994.245401] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8002.047598] loop0: detected capacity change from 0 to 204800000 [ 8002.163637] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8002.178225] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8002.417939] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8003.059220] loop0: detected capacity change from 0 to 204800000 [ 8003.176659] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8003.184711] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8003.395174] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8004.055672] loop0: detected capacity change from 0 to 204800000 [ 8004.174770] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8004.185991] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8004.249537] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8004.947907] loop0: detected capacity change from 0 to 204800000 [ 8005.046538] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8005.801483] loop0: detected capacity change from 0 to 204800000 [ 8005.822467] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8008.440718] loop0: detected capacity change from 0 to 204800000 [ 8008.463821] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8011.097603] loop0: detected capacity change from 0 to 204800000 [ 8011.123112] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8013.911364] loop0: detected capacity change from 0 to 204800000 [ 8013.927325] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8016.463895] loop0: detected capacity change from 0 to 204800000 [ 8016.520239] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8016.545161] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8016.608199] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8016.615749] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8016.636081] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8016.646521] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8016.654861] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8016.664389] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8016.675755] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8016.685462] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8016.698428] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8016.719574] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8016.729861] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8016.737207] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8016.744920] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8016.751093] Lustre: 279685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8020.931222] Lustre: server umount MGS complete [ 8021.531549] loop0: detected capacity change from 0 to 204800000 [ 8021.574600] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8021.919613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8021.958713] Lustre: lustre-MDT0000: Aborting client recovery [ 8022.540216] loop1: detected capacity change from 0 to 204800000 [ 8022.647079] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8022.693834] Lustre: 280119:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8022.700052] Lustre: 280119:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8022.707594] Lustre: 280119:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8022.714821] Lustre: 280119:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8022.723306] Lustre: 280119:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8022.730379] Lustre: 280119:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8022.737617] Lustre: 280119:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8022.753506] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8022.890923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8022.931522] Lustre: lustre-MDT0001: Aborting client recovery [ 8023.503114] loop2: detected capacity change from 0 to 204800000 [ 8023.605667] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8023.671882] Lustre: 280197:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8023.683607] Lustre: 280197:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8023.707883] Lustre: 280197:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8023.723788] Lustre: 280197:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8023.730865] Lustre: 280197:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8023.737806] Lustre: 280197:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8023.744910] Lustre: 280197:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8023.828333] Lustre: Found index 0 for lustre-OST0000, updating log [ 8023.853926] Lustre: 280215:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8023.898785] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8023.913155] Lustre: lustre-OST0000: Aborting recovery [ 8024.555078] loop3: detected capacity change from 0 to 204800000 [ 8024.621859] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8024.698852] Lustre: 280281:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8024.709487] Lustre: 280281:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8024.721446] Lustre: 280281:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8024.733551] Lustre: 280281:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8024.740977] Lustre: 280281:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8024.749857] Lustre: 280281:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8024.761063] Lustre: 280281:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8024.795735] Lustre: 280286:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8024.840694] Lustre: lustre-OST0001: Aborting recovery [ 8030.215736] LustreError: 280063:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8032.231060] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8032.234259] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8032.238794] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8032.245961] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8032.272479] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8032.291911] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8036.615286] LustreError: 280030:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 673us: rc = -2 [ 8037.116150] LustreError: 280031:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 565us: rc = -2 [ 8037.128819] LustreError: 280031:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 28 previous similar messages [ 8052.704557] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8052.705498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8052.714258] Lustre: Skipped 3 previous similar messages [ 8052.729260] Lustre: Skipped 3 previous similar messages [ 8055.113518] Lustre: server umount lustre-MDT0000 complete [ 8055.727452] LustreError: 280012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784591795 with bad export cookie 13210512147869119492 [ 8055.735037] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8057.825314] LustreError: 280028:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8057.825434] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8057.825874] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8057.839894] LustreError: 280028:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8057.870121] Lustre: Skipped 1 previous similar message [ 8060.831945] LustreError: 280027:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8060.831989] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8060.853561] Lustre: Skipped 2 previous similar messages [ 8065.952474] LustreError: 280028:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8065.952722] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8065.965028] LustreError: 280028:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8065.972989] Lustre: Skipped 1 previous similar message [ 8070.111351] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8070.338126] Lustre: server umount lustre-MDT0001 complete [ 8077.159610] Lustre: server umount lustre-OST0000 complete [ 8092.128847] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8092.256823] Lustre: server umount lustre-OST0001 complete [ 8097.504580] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 19:57:16 (1784591836) [ 8116.574318] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 8123.435607] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8144.894197] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8145.031637] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8145.056300] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8145.114757] Lustre: lustre-MDT0000: new disk, initializing [ 8145.171822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8145.180361] Lustre: Skipped 1 previous similar message [ 8145.192876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8147.640307] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8154.464689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8154.543726] Lustre: 284353:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8154.602177] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8154.606121] Lustre: Skipped 1 previous similar message [ 8154.652690] Lustre: lustre-MDT0001: new disk, initializing [ 8154.726657] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8154.752753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8154.763551] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8157.550595] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8163.759354] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8167.537429] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8171.820490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8171.966419] Lustre: lustre-OST0000: new disk, initializing [ 8171.969873] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8171.974220] Lustre: 286320:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8172.029858] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8175.726814] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8181.280770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8181.294680] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8181.377155] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8182.339362] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8186.155323] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8189.955441] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8195.674358] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8237.540843] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8237.551341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8237.555574] Lustre: Skipped 1 previous similar message [ 8240.771957] Lustre: server umount lustre-OST0000 complete [ 8246.755334] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8246.768989] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8246.780640] Lustre: Skipped 1 previous similar message [ 8246.785583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8249.312776] Lustre: server umount lustre-MDT0000 complete [ 8251.768658] LustreError: 284344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784591991 with bad export cookie 13210512147869122208 [ 8251.769505] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8251.775489] LustreError: 284344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8252.048759] Lustre: server umount lustre-MDT0001 complete [ 8257.760164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8260.463352] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8263.582741] Lustre: server umount MGS complete [ 8270.598513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8270.876730] LustreError: 290083:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8270.890395] LustreError: 290096:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8270.936692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8273.898165] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8275.936720] LustreError: 290084:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8279.249509] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8282.078915] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8286.570507] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8290.330279] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8294.919202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8298.930025] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8303.463685] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8319.976580] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8319.980185] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8319.987719] Lustre: Skipped 1 previous similar message [ 8319.989560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8320.001165] Lustre: Skipped 2 previous similar messages [ 8326.096399] Lustre: server umount lustre-OST0000 complete [ 8330.208541] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8330.222969] Lustre: Skipped 2 previous similar messages [ 8334.412417] Lustre: server umount lustre-MDT0000 complete [ 8335.328302] LustreError: 290084:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8335.347779] LustreError: 290084:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8336.703120] LustreError: 292804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784592076 with bad export cookie 13210512147869124035 [ 8336.705045] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8336.710360] LustreError: 292804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8336.913531] Lustre: server umount lustre-MDT0001 complete [ 8343.791527] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 8345.380182] Key type lgssc unregistered [ 8345.567118] LNet: 294031:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8345.576187] LNetError: 294031:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8346.596775] LNet: Removed LNI 192.168.203.135@tcp [ 8347.121309] Key type .llcrypt unregistered [ 8347.125842] Key type ._llcrypt unregistered [ 8368.500596] Key type ._llcrypt registered [ 8368.503288] Key type .llcrypt registered [ 8368.568890] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 8378.558313] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8379.175611] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8379.247578] alg: No test for adler32 (adler32-zlib) [ 8380.232100] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 8380.389708] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 8382.007143] Key type lgssc registered [ 8382.820549] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8403.839924] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8403.851437] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8405.010698] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8405.036131] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8405.092387] Lustre: lustre-MDT0000: new disk, initializing [ 8405.140273] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8405.160087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8407.968505] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8414.606524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8414.680975] Lustre: 297966:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8414.738526] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8414.743456] Lustre: Skipped 1 previous similar message [ 8414.827306] Lustre: lustre-MDT0001: new disk, initializing [ 8414.888463] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8414.917158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8414.924882] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8417.252991] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8423.417546] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8426.947488] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8431.018191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8431.239724] Lustre: lustre-OST0000: new disk, initializing [ 8431.246482] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8431.252895] Lustre: 299933:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8431.339079] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8434.977118] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8436.777963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8436.786679] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8436.864730] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8441.212745] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8447.004392] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8455.584976] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8462.712994] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8508.384513] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8508.389344] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8508.396542] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8508.897380] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8508.897404] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8508.915323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8513.650232] Lustre: server umount lustre-OST0000 complete [ 8517.599970] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8517.603379] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8517.608888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8524.259393] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8524.264107] Lustre: Skipped 3 previous similar messages [ 8528.352958] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8528.355631] Lustre: Skipped 1 previous similar message [ 8530.401541] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8530.472192] Lustre: server umount lustre-MDT0000 complete [ 8532.442278] LustreError: 301179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784592272 with bad export cookie 3854406266399381127 [ 8532.445039] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8532.451069] LustreError: 301179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8532.612845] Lustre: server umount lustre-MDT0001 complete [ 8536.908617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8539.078089] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8541.712332] Lustre: server umount MGS complete [ 8547.800401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8548.044879] LustreError: 303903:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8548.083210] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8550.344889] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8553.444770] LustreError: 303891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8553.452440] LustreError: 303891:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8554.736435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8554.951366] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8557.100341] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8561.130540] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8563.987839] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8567.612222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8567.814701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8571.058786] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8574.773434] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8591.330919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8591.331646] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8591.334086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8591.340558] Lustre: Skipped 2 previous similar messages [ 8594.735560] Lustre: server umount lustre-OST0000 complete [ 8601.056580] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8601.063207] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8602.887253] Lustre: server umount lustre-MDT0000 complete [ 8604.994290] LustreError: 303871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784592344 with bad export cookie 3854406266399382954 [ 8604.996537] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8605.002138] LustreError: 303871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8605.191779] Lustre: server umount lustre-MDT0001 complete [ 8610.743271] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 8612.261387] Key type lgssc unregistered [ 8612.415811] LNet: 307828:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8612.421421] LNetError: 307828:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8612.431704] LNet: Removed LNI 192.168.203.135@tcp [ 8612.881278] Key type .llcrypt unregistered [ 8612.884525] Key type ._llcrypt unregistered [ 8622.994040] Key type ._llcrypt registered [ 8622.995669] Key type .llcrypt registered [ 8623.052989] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 8630.675881] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8631.121211] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8631.190398] alg: No test for adler32 (adler32-zlib) [ 8632.126849] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 8632.302409] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 8633.911377] Key type lgssc registered [ 8634.428413] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8645.722256] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8653.759884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8653.770169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8654.899915] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8654.919381] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8654.977799] Lustre: lustre-MDT0000: new disk, initializing [ 8655.022856] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8655.033789] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8657.437556] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8663.729262] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8663.887437] Lustre: lustre-OST0000: new disk, initializing [ 8663.889829] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8663.894248] Lustre: Skipped 1 previous similar message [ 8663.899380] Lustre: 311851:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8663.948605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8667.487066] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8671.291442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8671.301613] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8671.330650] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9085.921975] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9085.928879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9087.413087] Lustre: server umount lustre-MDT0000 complete [ 9088.818209] LustreError: 311038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784592828 with bad export cookie 14420288769813560050 [ 9088.822865] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9088.879096] Lustre: server umount lustre-OST0000 complete [ 9098.954095] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 9103.064266] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9119.451332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9119.560947] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9119.579826] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9119.623340] Lustre: lustre-MDT0000: new disk, initializing [ 9119.663832] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9119.674903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9121.597383] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9126.871149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9126.941759] Lustre: 316112:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9126.973644] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9126.976566] Lustre: Skipped 1 previous similar message [ 9127.017927] Lustre: lustre-MDT0001: new disk, initializing [ 9127.068882] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9127.090312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9127.096345] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9128.983605] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9133.878964] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9136.094721] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9139.064593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9139.238612] Lustre: lustre-OST0000: new disk, initializing [ 9139.242691] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9139.248280] Lustre: 318079:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9139.308746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9141.951508] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9146.475514] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9146.887315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9146.892978] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9146.947124] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9148.315345] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9148.416728] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9150.184716] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9150.281592] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9151.970225] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9151.971183] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9151.973596] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9151.976745] Lustre: Skipped 1 previous similar message [ 9151.982308] LustreError: Skipped 1 previous similar message [ 9153.261953] Lustre: server umount lustre-OST0000 complete [ 9157.089916] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9157.091781] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9157.097435] Lustre: Skipped 1 previous similar message [ 9157.103142] Lustre: Skipped 2 previous similar messages [ 9162.208530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9162.213954] Lustre: Skipped 1 previous similar message [ 9167.328401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9167.332606] Lustre: Skipped 1 previous similar message [ 9169.375392] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9169.479560] Lustre: server umount lustre-MDT0000 complete [ 9171.354874] LustreError: 316103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784592911 with bad export cookie 14420288769816500841 [ 9171.355939] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9171.360133] LustreError: 316103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9171.511524] Lustre: server umount lustre-MDT0001 complete [ 9173.262233] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9174.082502] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 20:15:13 (1784592913) [ 9177.530872] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9177.807834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9180.111134] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9183.202375] LustreError: 320390:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9183.215056] LustreError: 320390:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9185.275843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9187.994791] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9192.122608] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9195.073634] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9198.616937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9198.823221] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9198.826329] Lustre: Skipped 1 previous similar message [ 9201.924687] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9205.460747] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9208.937142] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9209.030246] Lustre: lustre-OST0001: new disk, initializing [ 9209.033425] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9209.038785] Lustre: 323221:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9211.835658] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9216.581036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9216.590844] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9216.631454] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9216.913502] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9217.036099] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9217.699906] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9217.710593] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9218.726396] LustreError: 323976:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9218.734250] LustreError: 323976:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9219.963449] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8d5f08d6d800.ost_server_uuid 50 [ 9220.737448] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8d5f08d6d800.ost_server_uuid in FULL state after 0 sec [ 9220.772946] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9220.782690] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9223.195177] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8d5f08d6d800.ost_server_uuid 50 [ 9223.874198] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8d5f08d6d800.ost_server_uuid in FULL state after 0 sec [ 9224.874960] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9237.479201] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9237.482209] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9237.487879] Lustre: Skipped 1 previous similar message [ 9250.271458] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9250.321221] Lustre: server umount lustre-OST0001 complete [ 9250.564946] LustreError: 322211:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9252.837806] LustreError: 322209:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9252.849059] LustreError: 322209:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9254.371915] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9254.378291] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9254.389629] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9254.393637] Lustre: Skipped 7 previous similar messages [ 9257.440257] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9257.955930] LustreError: 322210:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9257.963093] LustreError: 322210:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9259.211509] Lustre: server umount lustre-OST0000 complete [ 9262.560279] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9262.564619] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9262.574806] Lustre: Skipped 1 previous similar message [ 9267.442091] Lustre: server umount lustre-MDT0000 complete [ 9268.192324] LustreError: 320390:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9268.200175] LustreError: 320390:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9269.271139] LustreError: 320371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784593009 with bad export cookie 14420288769816501709 [ 9269.277400] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9269.280382] LustreError: 320371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9269.464195] Lustre: server umount lustre-MDT0001 complete [ 9274.910930] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 9276.379846] Key type lgssc unregistered [ 9276.555640] LNet: 325903:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9276.559980] LNetError: 325903:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9276.568629] LNet: Removed LNI 192.168.203.135@tcp [ 9276.999384] Key type .llcrypt unregistered [ 9277.004564] Key type ._llcrypt unregistered [ 9288.797154] Key type ._llcrypt registered [ 9288.798541] Key type .llcrypt registered [ 9288.840086] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9289.352244] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9289.364993] alg: No test for adler32 (adler32-zlib) [ 9290.270434] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 9290.377793] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 9291.975160] Key type lgssc registered [ 9292.503763] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9296.191349] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9296.203197] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9297.421278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9299.129725] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9302.087465] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9304.703137] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9304.855852] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9306.444248] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9310.042974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9310.197240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9312.373853] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9314.276985] LustreError: 328981:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9314.286073] LustreError: 328981:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9315.153131] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9317.688519] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9317.770931] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9319.895587] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9322.561987] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9322.922493] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9322.922503] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9329.603909] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9329.700363] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9415.137832] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9415.143036] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9415.147877] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9415.153851] Lustre: Skipped 1 previous similar message [ 9418.528234] Lustre: server umount lustre-OST0001 complete [ 9420.260900] LustreError: 328982:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9420.271263] LustreError: 328982:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9421.792723] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9421.796463] LustreError: Skipped 1 previous similar message [ 9421.798168] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9421.803846] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9421.806456] Lustre: Skipped 1 previous similar message [ 9425.379237] LustreError: 328981:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9425.379636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9425.382243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9425.383518] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9425.394352] LustreError: 328981:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9425.397331] Lustre: Skipped 1 previous similar message [ 9427.041324] Lustre: server umount lustre-OST0000 complete [ 9430.497213] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9430.499660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9430.507918] Lustre: Skipped 1 previous similar message [ 9435.616503] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9435.619029] Lustre: Skipped 2 previous similar messages [ 9443.295189] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9443.376456] Lustre: server umount lustre-MDT0000 complete [ 9445.280176] LustreError: 327282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784593185 with bad export cookie 5230346796449744498 [ 9445.285703] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9445.290536] LustreError: 327282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9445.469692] Lustre: server umount lustre-MDT0001 complete [ 9450.917399] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 9452.110380] Key type lgssc unregistered [ 9452.253493] LNet: 332241:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9452.256648] LNetError: 332241:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9452.266317] LNet: Removed LNI 192.168.203.135@tcp [ 9452.609786] Key type .llcrypt unregistered [ 9452.611461] Key type ._llcrypt unregistered [ 9474.643412] Key type ._llcrypt registered [ 9474.648714] Key type .llcrypt registered [ 9474.719991] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9475.121513] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9475.176090] alg: No test for adler32 (adler32-zlib) [ 9476.106159] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 9476.230698] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 9477.855740] Key type lgssc registered [ 9478.359229] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9482.850530] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9482.859083] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9484.091156] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9485.924293] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9489.926194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9490.130365] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9491.974051] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9493.210338] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9495.964316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9496.156658] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9498.523529] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9499.240130] LustreError: 335992:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9499.252728] LustreError: 335992:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9502.181316] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9502.294194] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9504.614365] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9508.329397] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9508.330718] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9514.229423] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9520.958493] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9526.239796] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9526.241867] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9526.245904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9528.800048] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9528.803461] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9528.808148] Lustre: Skipped 2 previous similar messages [ 9530.715148] Lustre: server umount lustre-MDT0000 complete [ 9531.907281] LustreError: 337513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784593271 with bad export cookie 1473792076345497314 [ 9531.910967] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9531.911639] LustreError: 337513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9532.032561] Lustre: server umount lustre-MDT0001 complete [ 9544.128369] Lustre: server umount lustre-OST0000 complete [ 9555.611975] Lustre: server umount lustre-OST0001 complete [ 9561.251635] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9564.845860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9564.993743] LustreError: 339765:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9565.014644] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9566.513523] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9569.552517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9571.154348] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9572.152251] Lustre: 340904:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9574.406240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9574.550025] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9574.552940] Lustre: Skipped 1 previous similar message [ 9575.588078] LustreError: 341256:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9575.593218] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9575.594823] LustreError: 341256:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9576.513211] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9579.453733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9580.585267] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9581.575180] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9584.856304] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9586.196872] Lustre: 342776:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9590.240106] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9590.244050] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9590.251470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9590.254650] Lustre: Skipped 3 previous similar messages [ 9593.445600] Lustre: server umount lustre-MDT0000 complete [ 9594.884286] LustreError: 340507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784593334 with bad export cookie 1473792076345499001 [ 9594.887690] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9594.888666] LustreError: 340507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9595.018299] Lustre: server umount lustre-MDT0001 complete [ 9606.592574] Lustre: server umount lustre-OST0000 complete [ 9618.094205] Lustre: server umount lustre-OST0001 complete [ 9619.440870] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9620.032239] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9620.671023] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 20:22:40 (1784593360) [ 9628.765296] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9632.049549] loop0: detected capacity change from 0 to 1099511627776 [ 9632.247533] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9632.251516] *** VALIDATE xfs *** [ 9632.256328] XFS (loop0): Mounting V5 Filesystem [ 9632.266726] XFS (loop0): Ending clean mount [ 9632.791595] loop1: detected capacity change from 0 to 18691697672192 [ 9654.876376] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 9658.373932] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9670.641079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9670.723312] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9670.732899] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9670.767186] Lustre: lustre-MDT0000: new disk, initializing [ 9670.789775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9670.792995] Lustre: Skipped 1 previous similar message [ 9670.800428] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9672.158935] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9676.270057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9676.305664] Lustre: 347887:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9676.330075] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9676.332661] Lustre: Skipped 1 previous similar message [ 9676.364880] Lustre: lustre-MDT0001: new disk, initializing [ 9676.413323] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9676.416947] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9677.819597] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9681.525426] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9683.446490] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9685.589831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9685.686632] Lustre: lustre-OST0000: new disk, initializing [ 9685.688461] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9685.691174] Lustre: 349853:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9687.544879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9687.549229] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9687.589434] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9687.736708] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9691.412390] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9692.859237] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9692.931890] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9694.398365] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9694.479846] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9697.761573] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9697.761978] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9697.762503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9697.762508] Lustre: Skipped 3 previous similar messages [ 9697.764062] LustreError: Skipped 1 previous similar message [ 9697.768294] Lustre: Skipped 4 previous similar messages [ 9701.269656] Lustre: server umount lustre-OST0000 complete [ 9702.880637] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9702.887891] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9702.891136] Lustre: Skipped 1 previous similar message [ 9707.487936] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9713.120125] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9713.123206] Lustre: Skipped 3 previous similar messages [ 9717.215122] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9717.276115] Lustre: server umount lustre-MDT0000 complete [ 9718.721443] LustreError: 347878:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784593458 with bad export cookie 1473792076345500527 [ 9718.725299] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9718.727960] LustreError: 347878:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9718.854916] Lustre: server umount lustre-MDT0001 complete [ 9721.765554] XFS (loop0): Unmounting Filesystem [ 9723.534532] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 20:24:23 (1784593463) [ 9725.899609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9726.053133] LustreError: 352409:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9726.077621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9726.081308] Lustre: Skipped 2 previous similar messages [ 9727.570675] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9730.550519] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9732.182145] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9734.693596] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9736.604410] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9739.082138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9741.467300] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9744.382823] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9760.738358] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9760.739183] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9760.743295] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9760.749963] Lustre: Skipped 2 previous similar messages [ 9760.755647] Lustre: Skipped 3 previous similar messages [ 9764.443548] Lustre: server umount lustre-OST0000 complete [ 9780.191127] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9780.252946] Lustre: server umount lustre-MDT0000 complete [ 9781.216243] LustreError: 352410:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9781.222185] LustreError: 352410:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9781.580213] LustreError: 352434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784593521 with bad export cookie 1473792076345501416 [ 9781.583990] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9781.586463] LustreError: 352434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9781.707352] Lustre: server umount lustre-MDT0001 complete [ 9785.657288] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 9786.796457] Key type lgssc unregistered [ 9786.936356] LNet: 356060:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9786.938836] LNetError: 356060:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9786.951368] LNet: Removed LNI 192.168.203.135@tcp [ 9787.292223] Key type .llcrypt unregistered [ 9787.293587] Key type ._llcrypt unregistered [ 9798.226498] Key type ._llcrypt registered [ 9798.228606] Key type .llcrypt registered [ 9798.273456] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9798.742561] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9798.754544] alg: No test for adler32 (adler32-zlib) [ 9799.626693] Lustre: Lustre: Build Version: 2.17.54_183_ga497ed6 [ 9799.735175] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 9801.327176] Key type lgssc registered [ 9801.706369] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9804.667427] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9804.680639] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9805.865360] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9807.397634] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9810.393395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9810.550476] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9811.933883] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9814.346769] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9816.380424] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9818.980444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9819.121610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9821.424347] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9824.264276] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9839.884503] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.35@tcp (stopping) [ 9840.608435] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9840.611409] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9840.613916] Lustre: Skipped 1 previous similar message [ 9843.621847] Lustre: server umount lustre-MDT0001 complete [ 9844.993648] LustreError: 357599:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9845.001846] LustreError: 357599:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9845.728453] LustreError: 357602:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9845.737823] LustreError: 357602:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9847.768792] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9847.847491] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9847.853427] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9847.873498] Lustre: 357593:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9848.019636] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9849.679016] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9853.418112] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9853.927987] LustreError: 361281:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9853.931622] LustreError: 361281:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9853.935323] LustreError: 357595:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9853.957715] LustreError: 361284:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.135@tcp: cfg command failed: rc = -17 [ 9853.962969] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9853.962969] [10155.489488] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10155.489668] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10155.492278] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10155.495095] Lustre: Skipped 1 previous similar message [10155.498353] Lustre: Skipped 2 previous similar messages [10158.425727] Lustre: server umount lustre-MDT0001 complete [10160.609553] LustreError: 357621:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10160.616571] LustreError: 357621:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10161.992321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10162.035241] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10162.037040] Lustre: Found index 1 for lustre-MDT0001, updating log [10162.050359] Lustre: 357593:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10162.164643] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10163.504717] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10167.270293] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10170.853614] LustreError: 373956:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10170.856504] LustreError: 373956:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10170.859628] LustreError: 357595:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10170.862411] LustreError: 357595:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10170.877639] LustreError: 373959:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [10170.882062] LustreError: 373959:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [10170.885873] LustreError: 373959:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.135@tcp: cfg command failed: rc = -17 [10170.890693] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.135@tcp [10170.890693] [10469.347043] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10469.347264] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10469.352891] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10469.355901] Lustre: Skipped 2 previous similar messages [10472.338311] Lustre: server umount lustre-MDT0001 complete [10474.465203] LustreError: 357602:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10474.470474] LustreError: 357602:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10475.560117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10475.587616] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10475.589563] Lustre: Found index 1 for lustre-MDT0001, updating log [10475.600949] Lustre: 357591:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10475.693187] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10476.942113] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10481.126804] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10483.171969] LustreError: 386646:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10483.175374] LustreError: 386646:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10483.178625] LustreError: 357595:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10483.180213] LustreError: 357595:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10483.194804] LustreError: 386649:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10483.201304] LustreError: 386649:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.135@tcp: cfg command failed: rc = -114 [10483.205061] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10483.205061] [10793.442327] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10793.443623] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10793.445813] Lustre: Skipped 3 previous similar messages [10793.447960] Lustre: Skipped 2 previous similar messages [10793.450664] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10796.254992] Lustre: server umount lustre-MDT0000 complete [10797.464157] LustreError: 373329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784594537 with bad export cookie 8984252568718997648 [10797.468031] LustreError: 373329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10797.468896] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10798.559966] LustreError: 366915:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10798.560232] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10798.565229] LustreError: 366915:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10802.592182] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10802.594765] Lustre: Skipped 2 previous similar messages [10803.636813] Lustre: server umount lustre-MDT0001 complete [10804.932389] Lustre: server umount lustre-OST0000 complete [10808.046932] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [10811.045101] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [10821.987900] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 20:42:41 (1784594561) [10824.048333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10824.140295] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10824.151499] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10824.188493] Lustre: lustre-MDT0000: new disk, initializing [10824.209977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10824.216106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10825.568712] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10829.667328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10829.716923] Lustre: 401344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10829.741269] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10829.743245] Lustre: Skipped 1 previous similar message [10829.777482] Lustre: lustre-MDT0001: new disk, initializing [10829.808636] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10829.821314] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10829.824808] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10831.156210] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10834.844451] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10836.651043] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10838.859816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10838.972409] Lustre: lustre-OST0000: new disk, initializing [10838.974295] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10838.978272] Lustre: 403309:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10839.012080] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10839.475751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10839.478471] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10839.516400] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10841.039517] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10844.776110] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10849.762891] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10849.764220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10849.766692] Lustre: Skipped 2 previous similar messages [10849.768653] Lustre: Skipped 1 previous similar message [10853.595434] Lustre: server umount lustre-MDT0000 complete [10854.835063] LustreError: 401337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784594594 with bad export cookie 8984252568718999587 [10854.837371] LustreError: 402150:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [10854.838757] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10854.839203] LustreError: 401337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [10854.880786] LustreError: 401379:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10854.885551] LustreError: 401379:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10854.959286] Lustre: server umount lustre-MDT0001 complete [10862.422029] Lustre: server umount lustre-OST0000 complete [10871.606779] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [10874.866316] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [10885.137088] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10601 sec ========= 20:43:45 (1784594625) [10885.663094] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 20:43:45 (1784594625) === [10886.787948] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 20:43:46 (1784594626) === [10899.887478] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [10900.848669] Key type lgssc unregistered [10900.982275] LNet: 409659:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10900.984691] LNetError: 409659:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10900.992278] LNet: Removed LNI 192.168.203.135@tcp [10901.274868] Key type .llcrypt unregistered [10901.275925] Key type ._llcrypt unregistered