[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 488052902 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2576MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002260] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.005010] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008024] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010009] pid_max: default: 32768 minimum: 301 [ 0.011146] LSM: Security Framework initializing [ 0.012065] Yama: becoming mindful. [ 0.013041] SELinux: Initializing. [ 0.014091] *** VALIDATE selinux *** [ 0.022814] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027180] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028200] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029125] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030124] *** VALIDATE tmpfs *** [ 0.032397] *** VALIDATE proc *** [ 0.033257] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.035272] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036174] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037013] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038032] Spectre V2 : User space: Vulnerable [ 0.039012] Speculative Store Bypass: Vulnerable [ 0.041847] debug: unmapping init [mem 0xffffffffbd859000-0xffffffffbd860fff] [ 0.043195] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044714] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045026] ... version: 2 [ 0.046014] ... bit width: 48 [ 0.047015] ... generic registers: 4 [ 0.048012] ... value mask: 0000ffffffffffff [ 0.049013] ... max period: 00007fffffffffff [ 0.050012] ... fixed-purpose events: 3 [ 0.051012] ... event mask: 000000070000000f [ 0.052345] rcu: Hierarchical SRCU implementation. [ 0.054928] smp: Bringing up secondary CPUs ... [ 0.055611] x86: Booting SMP configuration: [ 0.056025] .... node #0, CPUs: #1 #2 #3 [ 0.059420] smp: Brought up 1 node, 4 CPUs [ 0.061015] smpboot: Max logical packages: 1 [ 0.062014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.141671] node 0 deferred pages initialised in 77ms [ 0.144600] devtmpfs: initialized [ 0.145162] x86/mm: Memory block size: 128MB [ 0.147534] gcov: version magic: 0x41383552 [ 0.149107] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.150055] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.151223] pinctrl core: initialized pinctrl subsystem [ 0.152143] [ 0.152916] ************************************************************* [ 0.153013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.154010] ** ** [ 0.155012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.156007] ** ** [ 0.157008] ** This means that this kernel is built to expose internal ** [ 0.158011] ** IOMMU data structures, which may compromise security on ** [ 0.159007] ** your system. ** [ 0.160008] ** ** [ 0.161011] ** If you see this message and you are not debugging the ** [ 0.162007] ** kernel, report this immediately to your vendor! ** [ 0.163009] ** ** [ 0.164008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.165007] ************************************************************* [ 0.166602] NET: Registered protocol family 16 [ 0.167307] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.168046] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.169044] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.171015] cpuidle: using governor menu [ 0.173843] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.177444] PCI: Using configuration type 1 for base access [ 0.180191] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.192050] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.193013] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.194157] cryptd: max_cpu_qlen set to 1000 [ 0.195241] ACPI: Added _OSI(Module Device) [ 0.196015] ACPI: Added _OSI(Processor Device) [ 0.197010] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.198010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.201196] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.203362] ACPI: Interpreter enabled [ 0.204045] ACPI: PM: (supports S0 S3 S4 S5) [ 0.205010] ACPI: Using IOAPIC for interrupt routing [ 0.206112] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.207285] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.215191] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.216026] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.217015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.218057] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.220082] acpiphp: Slot [2] registered [ 0.221084] acpiphp: Slot [5] registered [ 0.222015] acpiphp: Slot [6] registered [ 0.223090] acpiphp: Slot [7] registered [ 0.224116] acpiphp: Slot [8] registered [ 0.225080] acpiphp: Slot [9] registered [ 0.226099] acpiphp: Slot [10] registered [ 0.227078] acpiphp: Slot [3] registered [ 0.228015] acpiphp: Slot [4] registered [ 0.228998] acpiphp: Slot [11] registered [ 0.229947] acpiphp: Slot [12] registered [ 0.230073] acpiphp: Slot [13] registered [ 0.230960] acpiphp: Slot [14] registered [ 0.231080] acpiphp: Slot [15] registered [ 0.232102] acpiphp: Slot [16] registered [ 0.233078] acpiphp: Slot [17] registered [ 0.234048] acpiphp: Slot [18] registered [ 0.234946] acpiphp: Slot [19] registered [ 0.235047] acpiphp: Slot [20] registered [ 0.235903] acpiphp: Slot [21] registered [ 0.236073] acpiphp: Slot [22] registered [ 0.237084] acpiphp: Slot [23] registered [ 0.238024] acpiphp: Slot [24] registered [ 0.238814] acpiphp: Slot [25] registered [ 0.239088] acpiphp: Slot [26] registered [ 0.240090] acpiphp: Slot [27] registered [ 0.241047] acpiphp: Slot [28] registered [ 0.242100] acpiphp: Slot [29] registered [ 0.243078] acpiphp: Slot [30] registered [ 0.244060] acpiphp: Slot [31] registered [ 0.245058] PCI host bridge to bus 0000:00 [ 0.246023] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.247025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.248026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.249029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.250026] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.251027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.252180] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.254009] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.256088] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.265784] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.271066] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.273016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.275016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.277016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.280117] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.283721] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.286046] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.290021] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.295014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.308015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.313776] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.319040] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.325018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.330015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.346024] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.357421] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.370020] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.377016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.395017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.405710] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.414021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.423027] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.443020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.453536] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.463018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.469013] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.486016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.495673] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.502019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.508019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.526018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.534742] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.540015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.545015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.556017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.568259] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.570382] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.573381] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.575344] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.577218] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.581098] iommu: Default domain type: Passthrough [ 0.584551] SCSI subsystem initialized [ 0.586234] ACPI: bus type USB registered [ 0.588110] usbcore: registered new interface driver usbfs [ 0.591087] usbcore: registered new interface driver hub [ 0.593193] usbcore: registered new device driver usb [ 0.596190] pps_core: LinuxPPS API ver. 1 registered [ 0.599013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.603060] PTP clock support registered [ 0.606098] EDAC MC: Ver: 3.0.0 [ 0.610141] PCI: Using ACPI for IRQ routing [ 0.611890] NetLabel: Initializing [ 0.613013] NetLabel: domain hash size = 128 [ 0.615012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.617087] NetLabel: unlabeled traffic allowed by default [ 0.620044] vgaarb: loaded [ 0.621180] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.624017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.631488] clocksource: Switched to clocksource kvm-clock [ 0.736166] VFS: Disk quotas dquot_6.6.0 [ 0.737854] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.740440] *** VALIDATE ramfs *** [ 0.741639] *** VALIDATE hugetlbfs *** [ 0.742901] pnp: PnP ACPI init [ 0.744967] pnp: PnP ACPI: found 6 devices [ 0.761223] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.764473] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.766288] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.767875] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.769398] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.771140] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.773739] NET: Registered protocol family 2 [ 0.775701] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.780819] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.784505] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.789784] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.792734] TCP: Hash tables configured (established 65536 bind 65536) [ 0.795401] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.798094] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.800527] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.803059] NET: Registered protocol family 1 [ 0.805757] RPC: Registered named UNIX socket transport module. [ 0.809397] RPC: Registered udp transport module. [ 0.811312] RPC: Registered tcp transport module. [ 0.813265] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.815805] NET: Registered protocol family 44 [ 0.817337] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.819418] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.821092] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.823140] PCI: CLS 0 bytes, default 64 [ 0.824505] Unpacking initramfs... [ 2.251903] debug: unmapping init [mem 0xffff9d50fcc54000-0xffff9d50fffbffff] [ 2.255219] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.256860] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.259566] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.786758] Initialise system trusted keyrings [ 2.788558] Key type blacklist registered [ 2.790906] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.801662] zbud: loaded [ 2.805608] *** VALIDATE nfs *** [ 2.806461] *** VALIDATE nfs4 *** [ 2.807662] pstore: using deflate compression [ 2.809708] Platform Keyring initialized [ 2.926675] NET: Registered protocol family 38 [ 2.928795] Key type asymmetric registered [ 2.930659] Asymmetric key parser 'x509' registered [ 2.932880] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.936545] io scheduler mq-deadline registered [ 2.938771] io scheduler kyber registered [ 2.940853] io scheduler bfq registered [ 2.943207] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.946797] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.950420] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.953910] ACPI: Power Button [PWRF] [ 2.960136] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.967302] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.981615] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.003309] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.019472] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.048646] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.078118] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.082685] Non-volatile memory driver v1.3 [ 3.084324] Linux agpgart interface v0.103 [ 3.116596] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.118817] vda: detected capacity change from 0 to 74698752 [ 3.135912] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.139632] vdb: detected capacity change from 0 to 1073741824 [ 3.158245] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.161702] vdc: detected capacity change from 0 to 2621440000 [ 3.183114] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.185538] vdd: detected capacity change from 0 to 2621440000 [ 3.206119] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.208495] vde: detected capacity change from 0 to 4294967296 [ 3.224145] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.226397] vdf: detected capacity change from 0 to 4294967296 [ 3.233610] libphy: Fixed MDIO Bus: probed [ 3.239989] usbcore: registered new interface driver usbserial_generic [ 3.242634] usbserial: USB Serial support registered for generic [ 3.245257] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.249742] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.251722] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.254755] mousedev: PS/2 mouse device common for all mice [ 3.258333] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.262084] rtc_cmos 00:05: RTC can wake from S4 [ 3.265449] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.266089] rtc_cmos 00:05: registered as rtc0 [ 3.271249] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.274389] intel_pstate: CPU model not supported [ 3.276719] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.280622] hid: raw HID events driver (C) Jiri Kosina [ 3.283660] usbcore: registered new interface driver usbhid [ 3.285802] usbhid: USB HID core driver [ 3.287444] drop_monitor: Initializing network drop monitor service [ 3.289727] Initializing XFRM netlink socket [ 3.291792] NET: Registered protocol family 10 [ 3.294937] Segment Routing with IPv6 [ 3.296505] NET: Registered protocol family 17 [ 3.298568] mpls_gso: MPLS GSO support [ 3.304252] RAS: Correctable Errors collector initialized. [ 3.306558] AVX version of gcm_enc/dec engaged. [ 3.308571] AES CTR mode by8 optimization enabled [ 3.393393] sched_clock: Marking stable (3393342658, 0)->(4277548799, -884206141) [ 3.397181] registered taskstats version 1 [ 3.399106] Loading compiled-in X.509 certificates [ 3.401430] zswap: loaded using pool lzo/zbud [ 3.427381] Key type big_key registered [ 3.438250] Key type encrypted registered [ 3.439582] ima: No TPM chip found, activating TPM-bypass! [ 3.441591] ima: Allocated hash algorithm: sha1 [ 3.443288] ima: No architecture policies found [ 3.444943] evm: Initialising EVM extended attributes: [ 3.446704] evm: security.selinux [ 3.447830] evm: security.ima [ 3.449163] evm: security.capability [ 3.450490] evm: HMAC attrs: 0x1 [ 3.453127] rtc_cmos 00:05: setting system clock to 2026-08-12 19:58:43 UTC (1786564723) [ 3.459659] debug: unmapping init [mem 0xffffffffbe803000-0xffffffffbe9fffff] [ 3.463068] debug: unmapping init [mem 0xffffffffbd582000-0xffffffffbd858fff] [ 3.473071] Write protecting the kernel read-only data: 28672k [ 3.476388] debug: unmapping init [mem 0xffffffffbbc03000-0xffffffffbbdfffff] [ 3.479424] debug: unmapping init [mem 0xffffffffbc514000-0xffffffffbc5fffff] [ 3.513958] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.522393] systemd[1]: Detected virtualization kvm. [ 3.525227] systemd[1]: Detected architecture x86-64. [ 3.527128] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.551878] systemd[1]: No hostname configured. [ 3.553590] systemd[1]: Set hostname to . [ 3.555678] random: systemd: uninitialized urandom read (16 bytes read) [ 3.558383] systemd[1]: Initializing machine ID from random generator. [ 3.614376] random: ln: uninitialized urandom read (6 bytes read) [ 3.696182] random: systemd: uninitialized urandom read (16 bytes read) [ 3.698889] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.704313] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.708871] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.302420] device-mapper: uevent: version 1.0.3 [ 4.303836] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.981541] virtio_net virtio0 ens2: renamed from eth0 [ 4.984500] random: fast init done [ 5.024161] scsi host0: ata_piix [ 5.050889] scsi host1: ata_piix [ 5.053023] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.056430] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.270685] dracut-initqueue[588]: RTNETLINK answers: File exists [ 9.957942] random: crng init done [ 9.959419] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.263346] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.518714] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.776563] SELinux: Disabled at runtime. [ 11.842641] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.852425] systemd[1]: Detected virtualization kvm. [ 11.854420] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.344416] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.347743] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.353178] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.357259] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.362709] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.370650] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.380120] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ 12.423643] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Control Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.853314] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.209255] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.301069] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.399851] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.411781] EDAC sbridge: Ver: 1.1.2 [ 14.824592] Key type dns_resolver registered [ 15.122775] NFS: Registering the id_resolver key type [ 15.124837] Key type id_resolver registered [ 15.126189] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg212-server login: [ 52.809233] libcfs: loading out-of-tree module taints kernel. [ 52.890524] Key type ._llcrypt registered [ 52.896041] Key type .llcrypt registered [ 52.997594] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 76.292713] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 78.363142] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 78.390507] alg: No test for adler32 (adler32-zlib) [ 79.730032] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 80.696884] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 82.600193] Key type lgssc registered [ 84.690978] Lustre: Echo OBD driver; http://www.lustre.org/ [ 95.578305] hrtimer: interrupt took 14097935 ns [ 105.682133] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 153.578983] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 169.535200] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 169.608913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 170.929496] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 170.965695] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 171.115222] Lustre: lustre-MDT0000: new disk, initializing [ 171.231358] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 171.257080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 176.578436] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 192.323765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 192.493693] Lustre: 6522:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 192.540374] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 192.547730] Lustre: Skipped 1 previous similar message [ 192.673500] Lustre: lustre-MDT0001: new disk, initializing [ 192.742514] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 192.797066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 192.815330] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 197.792159] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 203.283624] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 214.076961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 214.373792] Lustre: lustre-OST0000: new disk, initializing [ 214.387589] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 214.400480] Lustre: 8460:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 214.542318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 218.007682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 218.023437] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 218.131973] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 220.712776] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 236.757666] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 236.970626] Lustre: lustre-OST0001: new disk, initializing [ 236.977174] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 236.984968] Lustre: 9532:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 237.098332] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 243.814468] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 243.840814] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 243.999381] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 244.011568] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.657601] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 262.922145] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 270.002834] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing check_logdir /tmp/testlogs/ [ 275.343904] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing yml_node [ 280.518565] Lustre: DEBUG MARKER: Client: 2.17.57.2 [ 283.361527] Lustre: DEBUG MARKER: MDS: 2.17.57.2 [ 286.062919] Lustre: DEBUG MARKER: OSS: 2.17.57.2 [ 287.548412] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 12 16:03:26 EDT 2026 [ 304.935484] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 306.932819] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 326.113379] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 326.116651] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 326.128175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 326.151803] Lustre: Skipped 1 previous similar message [ 326.165082] Lustre: Skipped 3 previous similar messages [ 330.209667] Lustre: server umount lustre-MDT0000 complete [ 334.516332] LustreError: 6513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565054 with bad export cookie 8217149890294809547 [ 334.535401] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 336.353761] LustreError: 6528:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 336.356885] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 336.361413] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 336.376566] LustreError: 6528:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 336.416227] Lustre: Skipped 3 previous similar messages [ 339.361061] LustreError: 6530:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 339.366339] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 339.379540] LustreError: 6530:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 339.388764] Lustre: Skipped 2 previous similar messages [ 341.062792] Lustre: server umount lustre-MDT0001 complete [ 345.171988] Lustre: server umount lustre-OST0000 complete [ 349.406179] Lustre: server umount lustre-OST0001 complete [ 357.147586] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 368.739092] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 407.852694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 408.053521] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 408.081551] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 408.164969] Lustre: lustre-MDT0000: new disk, initializing [ 408.235670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 408.249538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 413.004732] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 423.778076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 423.940961] Lustre: 15085:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 424.012323] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 424.019640] Lustre: Skipped 1 previous similar message [ 424.103494] Lustre: lustre-MDT0001: new disk, initializing [ 424.192934] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 424.227677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 424.233691] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 429.007383] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 438.382457] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 443.999712] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 450.997435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 451.275209] Lustre: lustre-OST0000: new disk, initializing [ 451.279390] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 451.286947] Lustre: 17056:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 451.372097] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 452.642982] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 452.667654] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 452.805900] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 457.642634] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 466.401696] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 471.478404] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 471.842610] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 477.097235] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 477.286969] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 483.815919] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 483.819243] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 483.822547] LustreError: Skipped 1 previous similar message [ 483.825105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 483.852451] Lustre: Skipped 1 previous similar message [ 485.319153] Lustre: server umount lustre-OST0000 complete [ 490.985442] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 491.002373] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 491.017530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 491.026857] Lustre: Skipped 1 previous similar message [ 495.434414] Lustre: server umount lustre-MDT0000 complete [ 499.171633] LustreError: 15093:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 499.208406] LustreError: 15093:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 499.723481] LustreError: 15077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565219 with bad export cookie 8217149890294811073 [ 499.728339] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 499.737633] LustreError: 15077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 500.057473] Lustre: server umount lustre-MDT0001 complete [ 505.278610] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:07:03 (1786565223) [ 512.920594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 513.477282] LustreError: 19320:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 513.609431] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 518.626569] LustreError: 19321:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 518.770267] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 528.866051] LustreError: 19320:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 528.892614] LustreError: 19320:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 529.110675] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 534.938558] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 543.777569] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 550.314489] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 557.060859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 565.165586] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 573.811438] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 578.060334] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 586.209837] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 586.216875] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 586.226765] Lustre: Skipped 1 previous similar message [ 586.241022] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 586.249353] Lustre: Skipped 2 previous similar messages [ 592.424098] Lustre: server umount lustre-MDT0000 complete [ 594.402592] LustreError: 19321:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 594.436659] LustreError: 19321:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 596.117520] LustreError: 19302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565316 with bad export cookie 8217149890294811955 [ 596.136341] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 596.144264] LustreError: 19302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 596.656740] Lustre: server umount lustre-MDT0001 complete [ 610.398807] Lustre: server umount lustre-OST0000 complete [ 631.529425] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 643.786528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 644.433285] LustreError: 23702:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 644.499324] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 644.511128] Lustre: Skipped 2 previous similar messages [ 649.018589] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 659.670045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 666.081207] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 669.742059] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 678.225123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 685.400983] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 695.765546] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 695.967244] Lustre: lustre-OST0001: new disk, initializing [ 695.973925] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 695.977809] Lustre: 26021:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 697.502892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 697.513629] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 697.719967] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 703.886456] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 714.086449] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 717.644821] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 732.136338] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 732.151814] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 732.170726] Lustre: Skipped 2 previous similar messages [ 732.179263] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 732.182653] Lustre: Skipped 3 previous similar messages [ 736.413447] Lustre: server umount lustre-MDT0000 complete [ 738.793608] LustreError: 23703:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 738.809497] LustreError: 23703:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 740.079114] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565460 with bad export cookie 8217149890294812809 [ 740.082407] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 740.092613] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 740.447938] Lustre: server umount lustre-MDT0001 complete [ 754.928218] Lustre: server umount lustre-OST0000 complete [ 759.267765] Lustre: 3639:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786565463/real 1786565463] req@ffff9d517d1e0e00 x1873348971971968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786565479 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 759.329878] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 759.359201] Lustre: Skipped 3 previous similar messages [ 760.013616] Lustre: server umount lustre-OST0001 complete [ 778.132466] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 789.913363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 790.441254] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 790.453978] Lustre: Skipped 3 previous similar messages [ 794.754966] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 806.146364] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 813.500782] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 817.474724] Lustre: 30187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 825.216432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 832.809883] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 842.574037] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 850.423759] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 858.855265] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 862.342349] Lustre: 32064:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 868.866148] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 868.874628] Lustre: Skipped 2 previous similar messages [ 868.882229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 868.887670] Lustre: Skipped 7 previous similar messages [ 875.412752] Lustre: server umount lustre-MDT0000 complete [ 879.077373] LustreError: 29040:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 879.111990] LustreError: 29040:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 879.412993] LustreError: 32267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565599 with bad export cookie 8217149890294814104 [ 879.420341] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 879.431703] LustreError: 32267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 879.894958] Lustre: server umount lustre-MDT0001 complete [ 894.124733] Lustre: server umount lustre-OST0000 complete [ 908.851147] Lustre: server umount lustre-OST0001 complete [ 916.881339] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:13:55 (1786565635) [ 944.218209] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 954.319033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.089582] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 968.131073] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 973.887533] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 977.050956] Lustre: 36020:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 983.730742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 990.400505] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 999.365827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1006.056224] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1012.882787] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1016.429615] Lustre: 37904:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1024.997026] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1025.005509] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1025.025078] Lustre: Skipped 1 previous similar message [ 1025.029916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1025.039398] Lustre: Skipped 4 previous similar messages [ 1028.131544] Lustre: server umount lustre-MDT0000 complete [ 1031.928453] LustreError: 34853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565751 with bad export cookie 8217149890294815210 [ 1031.942570] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1032.330635] Lustre: server umount lustre-MDT0001 complete [ 1046.189360] Lustre: server umount lustre-OST0000 complete [ 1059.483868] Lustre: server umount lustre-OST0001 complete [ 1069.471283] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:16:27 (1786565787) [ 1089.392353] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1099.520495] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1100.159890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1100.168576] Lustre: Skipped 7 previous similar messages [ 1105.039154] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1114.306781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1120.501915] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1124.411760] Lustre: 41195:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1132.277466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1139.952773] LustreError: 41548:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1140.000766] LustreError: 41548:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 1141.722918] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1151.479539] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1159.633748] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1168.715294] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1172.638269] Lustre: 43073:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1173.657570] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1173.665953] Lustre: Skipped 1 previous similar message [ 1177.571589] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1177.582868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1177.593903] Lustre: Skipped 4 previous similar messages [ 1177.607124] Lustre: Skipped 7 previous similar messages [ 1183.135179] Lustre: server umount lustre-MDT0000 complete [ 1187.073134] LustreError: 40029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565907 with bad export cookie 8217149890294816309 [ 1187.083215] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1187.091804] LustreError: 40029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1193.752856] Lustre: server umount lustre-MDT0001 complete [ 1201.935839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1212.385860] LustreError: 3638:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9d5170298380 x1873348972129280/t0(0) o250->MGC192.168.202.112@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1216.933955] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1222.143119] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1225.711642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1231.001099] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1231.352287] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1231.367293] Lustre: Skipped 1 previous similar message [ 1239.982843] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1242.160927] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1248.506907] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1250.511942] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1256.944461] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1261.966908] Lustre: server umount lustre-MDT0000 complete [ 1266.349469] LustreError: 40028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786565986 with bad export cookie 8217149890294817443 [ 1266.358135] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1266.361860] LustreError: 40028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1266.998190] Lustre: server umount lustre-MDT0001 complete [ 1281.807787] Lustre: server umount lustre-OST0000 complete [ 1283.682897] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786565987/real 1786565987] req@ffff9d517518d880 x1873348972164480/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786566003 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1285.874508] Lustre: server umount lustre-OST0001 complete [ 1294.035502] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:20:12 (1786566012) [ 1310.307592] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1319.369533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1324.648667] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1333.757490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1339.796341] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1342.800984] Lustre: 48833:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1349.316756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1356.890242] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1365.330423] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1372.529525] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1380.462353] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1384.855936] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1396.581683] Lustre: server umount lustre-MDT0000 complete [ 1400.154835] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566120 with bad export cookie 8217149890294818234 [ 1400.164628] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1400.171235] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1400.555716] Lustre: server umount lustre-MDT0001 complete [ 1414.807503] Lustre: server umount lustre-OST0000 complete [ 1417.717493] Lustre: 3640:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786566121/real 1786566121] req@ffff9d50432f2680 x1873348972216960/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786566137 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1418.508854] Lustre: server umount lustre-OST0001 complete [ 1428.005454] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:22:26 (1786566146) [ 1430.217151] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1431.937122] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:22:30 (1786566150) [ 1442.066592] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 1451.367885] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1473.654984] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1474.042377] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1474.164580] Lustre: lustre-MDT0000: new disk, initializing [ 1474.268279] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1479.020515] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1488.258554] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1495.510513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1495.791278] Lustre: lustre-OST0000: new disk, initializing [ 1495.797412] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1495.803579] Lustre: Skipped 1 previous similar message [ 1495.808694] Lustre: 54905:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1497.443089] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1497.464057] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1497.513608] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1502.313349] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1511.437439] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1513.457161] Lustre: DEBUG MARKER: create test files [ 1543.650475] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1543.666755] Lustre: Skipped 14 previous similar messages [ 1543.678092] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1543.681507] Lustre: Skipped 14 previous similar messages [ 1549.744546] Lustre: server umount lustre-OST0000 complete [ 1553.946298] Lustre: server umount lustre-MDT0000 complete [ 1565.036030] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 1567.974799] Key type lgssc unregistered [ 1568.302240] LNet: 56535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1568.317279] LNetError: 56535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1568.335156] LNet: Removed LNI 192.168.202.112@tcp [ 1569.372224] Key type .llcrypt unregistered [ 1569.374518] Key type ._llcrypt unregistered [ 1597.286270] Key type ._llcrypt registered [ 1597.295362] Key type .llcrypt registered [ 1597.434413] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1598.638365] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1598.785442] alg: No test for adler32 (adler32-zlib) [ 1599.852954] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 1600.073985] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 1601.768146] Key type lgssc registered [ 1603.018124] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1610.756727] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1610.770903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1612.244558] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1616.881993] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1623.564805] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1631.461652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1631.880131] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1637.042567] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1637.099905] Lustre: 59279:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1637.107767] Lustre: 59279:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1637.111796] Lustre: 59279:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1637.116786] Lustre: 59279:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1637.130816] Lustre: 59279:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1637.138059] Lustre: 59279:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1637.907540] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1643.935090] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1650.024644] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1654.648995] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1662.951428] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1662.955326] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1662.994866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1663.247459] Lustre: server umount lustre-OST0000 complete [ 1667.715658] Lustre: server umount lustre-MDT0000 complete [ 1678.749896] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 1681.508762] Key type lgssc unregistered [ 1681.904119] LNet: 60892:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1681.917480] LNetError: 60892:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1681.948620] LNet: Removed LNI 192.168.202.112@tcp [ 1683.158749] Key type .llcrypt unregistered [ 1683.161630] Key type ._llcrypt unregistered [ 1713.861323] Key type ._llcrypt registered [ 1713.864379] Key type .llcrypt registered [ 1713.941941] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1714.887095] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1715.020263] alg: No test for adler32 (adler32-zlib) [ 1716.098109] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 1716.316478] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 1717.984995] Key type lgssc registered [ 1719.193340] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1727.164247] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1727.185118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1728.590331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1732.768022] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1740.278897] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1746.831047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1747.182285] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1753.341047] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1753.425296] Lustre: 63729:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1753.433657] Lustre: 63729:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1753.437929] Lustre: 63729:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1753.442477] Lustre: 63729:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1753.452655] Lustre: 63729:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1753.459835] Lustre: 63729:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1753.726110] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1761.190534] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1766.511502] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1774.050261] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1774.064074] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1774.090669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1779.173869] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1779.694762] Lustre: server umount lustre-OST0000 complete [ 1783.694027] Lustre: server umount lustre-MDT0000 complete [ 1793.781376] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 1796.628919] Key type lgssc unregistered [ 1796.943831] LNet: 65287:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1796.953322] LNetError: 65287:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1796.984857] LNet: Removed LNI 192.168.202.112@tcp [ 1797.783345] Key type .llcrypt unregistered [ 1797.786606] Key type ._llcrypt unregistered [ 1816.225984] Key type ._llcrypt registered [ 1816.228177] Key type .llcrypt registered [ 1816.379867] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 1832.335089] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1833.362317] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1833.438876] alg: No test for adler32 (adler32-zlib) [ 1834.470332] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 1834.656650] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 1836.312337] Key type lgssc registered [ 1837.202657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1867.249711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1867.291162] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1868.440122] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1868.460876] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1868.513372] Lustre: lustre-MDT0000: new disk, initializing [ 1868.554042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1868.563484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1871.858732] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1880.197627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1880.280186] Lustre: 68434:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1880.329469] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1880.334964] Lustre: Skipped 1 previous similar message [ 1880.394271] Lustre: lustre-MDT0001: new disk, initializing [ 1880.458801] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1880.486921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1880.502649] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1884.550224] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1892.641666] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1897.339405] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1902.518224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1902.720299] Lustre: lustre-OST0000: new disk, initializing [ 1902.723841] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1902.728925] Lustre: 70404:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1902.810226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1907.918705] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1908.769941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1908.780627] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1908.923982] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1915.973792] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1920.028609] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1920.313917] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1923.939620] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1924.072617] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1929.191212] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1929.199216] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1929.213651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1929.700388] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1929.706665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1929.716603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1931.585811] Lustre: server umount lustre-OST0000 complete [ 1934.818381] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1934.834830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1934.843619] Lustre: Skipped 1 previous similar message [ 1939.944120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1939.947305] Lustre: Skipped 1 previous similar message [ 1940.636859] Lustre: server umount lustre-MDT0000 complete [ 1943.559953] LustreError: 68426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566663 with bad export cookie 12893440758372452240 [ 1943.563236] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1943.568487] LustreError: 68426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1943.870644] Lustre: server umount lustre-MDT0001 complete [ 1950.049234] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:31:09 (1786566669) [ 1957.862537] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1985.489996] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 1993.500460] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 2018.990458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2019.226956] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2019.249641] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2019.306793] Lustre: lustre-MDT0000: new disk, initializing [ 2019.380912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2019.392092] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2021.705707] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2028.389836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2028.466538] Lustre: 76415:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2028.505991] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2028.509421] Lustre: Skipped 1 previous similar message [ 2028.584644] Lustre: lustre-MDT0001: new disk, initializing [ 2028.657444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2028.672293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2028.681679] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2031.546719] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2037.473514] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2040.771582] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2044.652165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2044.823390] Lustre: lustre-OST0000: new disk, initializing [ 2044.826741] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2044.832731] Lustre: 78380:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2044.880715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2045.521493] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2045.530952] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2045.577631] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2048.230736] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2054.225952] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2057.274937] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2057.406192] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2059.850365] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2059.978474] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2061.282306] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2061.286561] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2061.294629] LustreError: Skipped 1 previous similar message [ 2061.302633] Lustre: Skipped 1 previous similar message [ 2061.304312] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2061.315190] Lustre: Skipped 1 previous similar message [ 2067.369779] Lustre: server umount lustre-OST0000 complete [ 2069.985660] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2069.990027] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2069.997562] Lustre: Skipped 1 previous similar message [ 2070.000432] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2070.008463] Lustre: Skipped 2 previous similar messages [ 2075.811144] Lustre: server umount lustre-MDT0000 complete [ 2078.268527] LustreError: 76408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566798 with bad export cookie 12893440758372453122 [ 2078.269248] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2078.275520] LustreError: 76408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2078.472879] Lustre: server umount lustre-MDT0001 complete [ 2083.860549] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:33:22 (1786566802) [ 2088.377756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2088.629350] LustreError: 80791:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2088.640929] LustreError: 80791:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2088.679675] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2091.577322] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2094.048909] LustreError: 80792:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2097.139241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2100.047271] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2104.246464] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2107.764288] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2112.272370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2116.559395] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2121.055279] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2123.277441] Lustre: 83457:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting bd827ceb-c1b9-4f30-bf6f-b6093fbb4abd at adminstrative request [ 2127.844549] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2127.855311] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2137.062882] Lustre: MGS: Client bd827ceb-c1b9-4f30-bf6f-b6093fbb4abd (at 0@lo) reconnecting [ 2137.068711] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xb2eeb40eaf554e74 to 0xb2eeb40eaf555114 [ 2137.079705] Lustre: MGC192.168.202.112@tcp: Connection restored to 0@lo (at 0@lo) [ 2157.043706] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2157.176606] Lustre: lustre-OST0001: new disk, initializing [ 2157.178554] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2157.181703] Lustre: 83860:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2157.238587] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2157.241435] Lustre: Skipped 2 previous similar messages [ 2160.519633] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2166.443604] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2166.845589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2166.857690] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2166.917578] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2171.877552] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2171.882683] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2171.890566] Lustre: Skipped 2 previous similar messages [ 2171.894794] Lustre: Skipped 2 previous similar messages [ 2174.254725] Lustre: server umount lustre-OST0001 complete [ 2177.002375] LustreError: 82609:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2177.009381] LustreError: 82609:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2182.118599] LustreError: 83499:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2182.127703] LustreError: 83499:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2182.825342] Lustre: server umount lustre-OST0000 complete [ 2191.573446] Lustre: server umount lustre-MDT0000 complete [ 2192.354211] LustreError: 80792:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2192.367368] LustreError: 80792:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2193.943834] LustreError: 83859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786566913 with bad export cookie 12893440758372454676 [ 2193.945749] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2193.949467] LustreError: 83859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2194.112867] Lustre: server umount lustre-MDT0001 complete [ 2199.066520] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:35:18 (1786566918) [ 2199.971144] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2201.055861] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:35:20 (1786566920) [ 2202.274219] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2203.699972] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:35:22 (1786566922) [ 2204.749512] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2205.926936] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:35:24 (1786566924) [ 2206.912259] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2207.875590] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:35:26 (1786566926) [ 2208.789731] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2212.126763] loop: module loaded [ 2212.140117] loop0: detected capacity change from 0 to 4294967296 [ 2212.169036] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2212.894961] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2214.556824] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2216.581439] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2230.264632] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2230.286341] LustreError: 87977:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2230.291662] LustreError: 87977:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2230.295335] LustreError: 87977:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2230.301293] LustreError: 87977:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2242.106265] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 2248.120091] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 2296.839817] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2297.126552] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2297.155152] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2297.329340] Lustre: lustre-MDT0000: new disk, initializing [ 2297.445027] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2297.455538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2304.350114] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2319.628611] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2319.815114] Lustre: 90843:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2320.195910] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2325.806719] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2336.454510] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2342.418944] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2348.309861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2348.554673] Lustre: 92814:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2354.314968] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2357.371278] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2357.520058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2362.686200] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2367.078385] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2367.283091] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2371.050491] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2371.224617] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2373.089300] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2373.096281] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2373.109806] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2373.129501] Lustre: Skipped 5 previous similar messages [ 2373.136558] Lustre: Skipped 7 previous similar messages [ 2379.109817] Lustre: server umount lustre-OST0000 complete [ 2388.939722] Lustre: server umount lustre-MDT0000 complete [ 2392.061342] LustreError: 90833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786567112 with bad export cookie 12893440758372455292 [ 2392.064149] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2392.071478] LustreError: 90833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2392.372086] Lustre: server umount lustre-MDT0001 complete [ 2400.781948] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:38:39 (1786567119) [ 2407.255109] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2407.788499] LustreError: 95226:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2412.078882] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2420.467191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2424.947470] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2431.854799] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2437.829909] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2444.559966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2444.781665] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2444.786726] Lustre: Skipped 4 previous similar messages [ 2450.337243] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2456.754074] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2460.517423] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2460.723417] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2464.897633] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2465.099453] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2471.565760] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2471.727275] Lustre: lustre-OST0001: new disk, initializing [ 2471.730925] Lustre: Skipped 2 previous similar messages [ 2471.735592] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2471.739669] Lustre: Skipped 3 previous similar messages [ 2471.744604] Lustre: 98597:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2477.106362] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2481.253298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2481.268259] Lustre: Skipped 2 previous similar messages [ 2481.273151] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2481.352925] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2486.593519] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2490.878618] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2491.102720] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2495.607286] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2495.867690] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2514.053842] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2535.802099] Lustre: Failing over lustre-MDT0000 [ 2536.341863] Lustre: server umount lustre-MDT0000 complete [ 2537.966047] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2537.973821] Lustre: Skipped 2 previous similar messages [ 2537.981374] LustreError: 96251:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2538.001101] LustreError: 96251:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 2554.341155] Lustre: 66661:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786567258/real 1786567258] req@0000000020a2ffca x1873350811171328/t0(0) o400->MGC192.168.202.112@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786567274 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2554.367570] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2561.111336] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2561.117140] LDISKFS-fs (dm-0): recovery complete [ 2561.123960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2565.181019] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2569.764775] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2569.766028] Lustre: 95222:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2570.223040] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2570.311764] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2570.334891] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 2 previous similar messages [ 2570.392551] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2570.736402] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2570.790585] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2571.058696] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2571.065568] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2571.384190] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2571.392326] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2571.400900] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2571.410036] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2572.371711] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2572.386749] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 2572.713468] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2572.722301] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 2573.683853] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2573.695352] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2574.673711] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2574.689421] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2575.025715] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2575.046940] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2577.908196] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2577.922297] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2578.901103] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2578.915926] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2579.232344] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2579.236028] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2586.005563] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2586.018605] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2586.985987] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2587.012334] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2587.336177] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2587.341784] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2602.018717] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2602.027890] Lustre: 101446:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 49 previous similar messages [ 2603.307285] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2603.323846] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2603.648108] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2603.652900] LustreError: 101446:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2629.104137] Lustre: 101446:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2629.112423] Lustre: 101446:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2629.117614] Lustre: 101446:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 5dd54382-8c2b-4978-8dc0-0bcb1bceb5f5@192.168.202.12@tcp [ 2629.125577] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2629.130875] Lustre: 101446:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@000000000d0ac5d7 x1873350795269248/t0(8589934780) o35->5dd54382-8c2b-4978-8dc0-0bcb1bceb5f5@192.168.202.12@tcp:289/0 lens 392/0 e 0 to 0 dl 1786567359 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2629.153305] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2629.168928] Lustre: lustre-MDT0000: Denying connection for new client 5dd54382-8c2b-4978-8dc0-0bcb1bceb5f5 (at 192.168.202.12@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2629.182907] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2629.216772] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2629.228194] Lustre: Skipped 2 previous similar messages [ 2629.228538] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2629.275290] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2629.276993] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2645.476366] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2645.485223] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2645.498309] Lustre: Skipped 1 previous similar message [ 2645.505647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2645.512314] Lustre: Skipped 7 previous similar messages [ 2649.740528] Lustre: server umount lustre-OST0000 complete [ 2652.154345] LustreError: 97525:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2652.177599] LustreError: 97525:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 36 previous similar messages [ 2659.903806] Lustre: server umount lustre-OST0001 complete [ 2665.442323] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2665.448577] LustreError: Skipped 3 previous similar messages [ 2669.903929] Lustre: server umount lustre-MDT0000 complete [ 2673.632358] LustreError: 95208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786567393 with bad export cookie 12893440758372506504 [ 2673.634249] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2673.638766] LustreError: 95208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2674.010465] Lustre: server umount lustre-MDT0001 complete [ 2682.639379] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:43:20 (1786567400) [ 2691.373155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2691.381438] Lustre: *** cfs_fail_loc=197, val=0*** [ 2707.043923] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2712.224541] LustreError: 103424:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.112@tcp: failed processing log, type 4: rc = -110 [ 2717.399076] Lustre: *** cfs_fail_loc=197, val=0*** [ 2717.407151] Lustre: Skipped 65 previous similar messages [ 2738.017093] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2738.022481] Lustre: Skipped 2 previous similar messages [ 2744.667693] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2752.579252] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2754.610189] Lustre: server umount lustre-OST0000 complete [ 2764.875741] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:44:42 (1786567482) [ 2789.386579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2790.357760] LustreError: 105043:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2790.399096] LustreError: 105043:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2795.504456] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2806.975313] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2811.873680] LustreError: 105074:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2811.893285] LustreError: 105074:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2811.909244] LustreError: 105074:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 2812.625205] Lustre: server umount lustre-MDT0000 complete [ 2850.350702] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 2863.264631] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 2904.726363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2905.133310] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2905.165389] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2905.301666] Lustre: lustre-MDT0000: new disk, initializing [ 2905.392937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2911.511475] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2923.670369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2923.889738] Lustre: 109251:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2924.235438] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2928.656635] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2940.520162] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2946.732040] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2953.879785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2954.072313] Lustre: 111222:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2961.293810] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2962.909672] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2974.886638] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2980.670681] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2980.836983] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2986.055885] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2986.284658] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2992.096943] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2992.110144] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2992.113967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2992.114637] Lustre: Skipped 10 previous similar messages [ 2992.127113] Lustre: Skipped 6 previous similar messages [ 2994.879975] Lustre: server umount lustre-OST0000 complete [ 3005.608378] Lustre: server umount lustre-MDT0000 complete [ 3007.457130] LustreError: 109258:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3007.481971] LustreError: 109258:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3009.952517] LustreError: 109245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786567729 with bad export cookie 12893440758372517207 [ 3009.953505] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3009.967485] LustreError: 109245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3010.483836] Lustre: server umount lustre-MDT0001 complete [ 3020.523916] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:48:58 (1786567738) [ 3030.684778] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 3033.681376] Key type lgssc unregistered [ 3033.901912] LNet: 113722:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3033.915109] LNetError: 113722:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3033.956881] LNet: Removed LNI 192.168.202.112@tcp [ 3035.022810] Key type .llcrypt unregistered [ 3035.029800] Key type ._llcrypt unregistered [ 3057.796279] Key type ._llcrypt registered [ 3057.799876] Key type .llcrypt registered [ 3057.958430] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 3075.452708] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3076.434925] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3076.471882] alg: No test for adler32 (adler32-zlib) [ 3077.669308] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 3078.020205] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 3079.832167] Key type lgssc registered [ 3081.185392] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3121.870801] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3121.888618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3123.117037] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3123.138395] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3123.186222] Lustre: lustre-MDT0000: new disk, initializing [ 3123.236547] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3123.247302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3127.325450] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3138.047514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3138.286438] Lustre: lustre-OST0000: new disk, initializing [ 3138.289898] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3138.293166] Lustre: Skipped 1 previous similar message [ 3138.296531] Lustre: 117896:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3138.352167] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3144.993854] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3146.312600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3146.323141] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3146.367386] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3155.379263] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3155.619783] Lustre: lustre-OST0001: new disk, initializing [ 3155.625511] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3155.635885] Lustre: 118761:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3155.745846] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3161.099034] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3161.105050] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3161.134960] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3162.206703] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3171.303939] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3171.312124] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3171.323225] Lustre: Skipped 1 previous similar message [ 3171.337820] Lustre: Skipped 1 previous similar message [ 3176.424288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3176.436881] Lustre: Skipped 1 previous similar message [ 3176.822792] Lustre: server umount lustre-MDT0000 complete [ 3183.575216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3187.940918] LustreError: 117085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786567907 with bad export cookie 10298143506989241292 [ 3187.949042] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3187.958373] LustreError: 117085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3188.013051] Lustre: server umount lustre-OST0000 complete [ 3191.675821] Lustre: server umount lustre-OST0001 complete [ 3200.137611] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 16:51:58 (1786567918) [ 3221.789746] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 16:52:20 (1786567940) [ 3231.411487] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3254.552876] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 16:52:52 (1786567972) [ 3294.247576] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 3305.103548] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3342.136786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3342.328700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3342.347749] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3342.401795] Lustre: lustre-MDT0000: new disk, initializing [ 3342.461231] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3342.474526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3347.302698] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3358.308989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3358.429579] Lustre: 126511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3358.504748] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3358.510482] Lustre: Skipped 1 previous similar message [ 3358.610551] Lustre: lustre-MDT0001: new disk, initializing [ 3358.737886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3358.783730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3358.800278] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3363.378041] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3372.579936] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3379.222490] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3385.898845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3386.179450] Lustre: lustre-OST0000: new disk, initializing [ 3386.185752] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3386.192129] Lustre: 128481:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3386.282415] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3392.156126] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3393.617437] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3393.625452] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3393.754801] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3402.975436] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3407.890133] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3408.144969] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3412.609667] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3412.930162] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3419.619780] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3419.624468] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3419.628773] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3419.634783] Lustre: Skipped 1 previous similar message [ 3424.743021] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3424.750151] Lustre: Skipped 1 previous similar message [ 3429.344082] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3429.644775] Lustre: server umount lustre-OST0000 complete [ 3434.992870] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3435.005820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3435.017307] Lustre: Skipped 1 previous similar message [ 3435.032096] Lustre: Skipped 1 previous similar message [ 3439.899529] Lustre: server umount lustre-MDT0000 complete [ 3443.760464] LustreError: 126502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568163 with bad export cookie 10298143506989242517 [ 3443.766444] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3443.775241] LustreError: 126502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3444.167430] Lustre: server umount lustre-MDT0001 complete [ 3454.041415] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 16:56:12 (1786568172) [ 3460.935508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3461.468812] LustreError: 130894:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3461.497863] LustreError: 130894:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3461.559188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3466.624807] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3466.732532] LustreError: 130895:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3471.842677] LustreError: 130894:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3475.907988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3481.130625] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3488.842467] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3494.116417] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3500.406410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3500.775545] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3500.778735] Lustre: Skipped 1 previous similar message [ 3507.838342] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3516.128652] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3521.169295] Lustre: *** cfs_fail_loc=159, val=0*** [ 3521.175920] Lustre: Skipped 1 previous similar message [ 3537.536967] Lustre: lustre-MDT0000: Client 84f66777-733b-4912-ac12-d349d5c8a1b9 (at 192.168.202.12@tcp) reconnecting [ 3539.576547] Lustre: *** cfs_fail_loc=159, val=0*** [ 3539.579214] Lustre: Skipped 4 previous similar messages [ 3555.992091] Lustre: lustre-MDT0000: Client 84f66777-733b-4912-ac12-d349d5c8a1b9 (at 192.168.202.12@tcp) reconnecting [ 3573.731104] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3573.735624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3573.746706] LustreError: Skipped 1 previous similar message [ 3573.748101] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3573.759638] Lustre: Skipped 1 previous similar message [ 3573.764160] Lustre: Skipped 1 previous similar message [ 3578.864400] LustreError: 132716:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3578.879498] LustreError: 132716:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3579.101404] Lustre: server umount lustre-OST0000 complete [ 3583.969897] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3583.977348] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3583.986868] Lustre: Skipped 1 previous similar message [ 3594.214440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3594.221463] Lustre: Skipped 6 previous similar messages [ 3597.280222] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3597.625581] Lustre: server umount lustre-MDT0000 complete [ 3599.330639] LustreError: 131898:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3599.340463] LustreError: 131898:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3600.577452] LustreError: 130876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568320 with bad export cookie 10298143506989243399 [ 3600.587561] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3600.858200] Lustre: server umount lustre-MDT0001 complete [ 3610.191362] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 3612.782649] Key type lgssc unregistered [ 3613.071758] LNet: 134678:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3613.082974] LNetError: 134678:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3613.099750] LNet: Removed LNI 192.168.202.112@tcp [ 3613.880722] Key type .llcrypt unregistered [ 3613.883780] Key type ._llcrypt unregistered [ 3635.528936] Key type ._llcrypt registered [ 3635.531614] Key type .llcrypt registered [ 3635.623523] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3636.662113] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3636.718211] alg: No test for adler32 (adler32-zlib) [ 3637.718880] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 3638.022996] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 3639.696200] Key type lgssc registered [ 3640.721214] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3648.521534] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3648.548729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3649.894547] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3654.079124] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3659.240922] LustreError: 136095:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3659.255247] LustreError: 136095:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3661.695639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3662.041621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3665.983881] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3672.320245] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3677.681326] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3683.942086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3684.386819] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3690.027689] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3694.643457] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3696.276083] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3703.904107] Lustre: *** cfs_fail_loc=159, val=0*** [ 3719.271334] Lustre: lustre-MDT0000: Client c488c49f-9d27-4729-9804-e34da65c608f (at 192.168.202.12@tcp) reconnecting [ 3721.280604] Lustre: *** cfs_fail_loc=159, val=0*** [ 3737.691706] Lustre: lustre-MDT0001: Client c488c49f-9d27-4729-9804-e34da65c608f (at 192.168.202.12@tcp) reconnecting [ 3738.894510] Lustre: *** cfs_fail_loc=159, val=0*** [ 3738.899547] Lustre: Skipped 3 previous similar messages [ 3755.111858] Lustre: lustre-MDT0001: Client c488c49f-9d27-4729-9804-e34da65c608f (at 192.168.202.12@tcp) reconnecting [ 3759.515869] Lustre: 139140:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3761.717339] Lustre: *** cfs_fail_loc=159, val=0*** [ 3761.722600] Lustre: Skipped 3 previous similar messages [ 3778.114690] Lustre: lustre-MDT0000: Client ea562476-eedf-4d73-b860-dc47f4d2ae2f (at 192.168.202.12@tcp) reconnecting [ 3779.515381] Lustre: *** cfs_fail_loc=159, val=0*** [ 3779.522968] Lustre: Skipped 14 previous similar messages [ 3796.032760] Lustre: lustre-MDT0000: Client ea562476-eedf-4d73-b860-dc47f4d2ae2f (at 192.168.202.12@tcp) reconnecting [ 3797.136605] Lustre: 139430:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3816.446402] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3816.453980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3816.456076] Lustre: Skipped 1 previous similar message [ 3816.462786] Lustre: Skipped 1 previous similar message [ 3821.017427] Lustre: server umount lustre-OST0000 complete [ 3826.146754] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3826.162794] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3826.181522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3830.221193] Lustre: server umount lustre-MDT0000 complete [ 3831.787979] LustreError: 139379:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3831.816462] LustreError: 139379:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3833.372043] LustreError: 136077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568553 with bad export cookie 8648818684326508266 [ 3833.375771] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3833.380470] LustreError: 136077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3833.577511] Lustre: server umount lustre-MDT0001 complete [ 3842.604575] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 3845.026380] Key type lgssc unregistered [ 3845.264993] LNet: 140510:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3845.275369] LNetError: 140510:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3845.291680] LNet: Removed LNI 192.168.202.112@tcp [ 3845.940315] Key type .llcrypt unregistered [ 3845.942606] Key type ._llcrypt unregistered [ 3864.003572] Key type ._llcrypt registered [ 3864.004849] Key type .llcrypt registered [ 3864.069543] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3864.732152] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3864.780467] alg: No test for adler32 (adler32-zlib) [ 3865.734179] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 3865.907058] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 3867.585392] Key type lgssc registered [ 3868.304656] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3873.310225] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3873.323231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3874.592842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3877.347978] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3882.683139] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3882.967702] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3886.101782] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3890.862269] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3894.925291] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3900.019454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3900.257349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3905.242572] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3907.439545] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3911.657850] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3919.739398] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3926.430755] Lustre: 144507:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3928.581612] Lustre: 144555:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3929.613823] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3930.330368] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3933.286354] Lustre: 144702:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3934.026181] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3934.801761] Lustre: 144808:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3938.273106] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3938.277401] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3938.284232] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3940.322641] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3940.330107] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3940.338098] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3940.362559] Lustre: Skipped 1 previous similar message [ 3944.609300] Lustre: server umount lustre-OST0000 complete [ 3949.548774] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3949.560868] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3949.577635] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3955.681325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3955.686819] Lustre: Skipped 3 previous similar messages [ 3960.803178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3960.814959] Lustre: Skipped 1 previous similar message [ 3962.336379] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3962.694519] Lustre: server umount lustre-MDT0000 complete [ 3965.924732] LustreError: 141900:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3965.943536] LustreError: 141900:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3966.075808] LustreError: 141887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568686 with bad export cookie 7677420223023150869 [ 3966.084312] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3966.097311] LustreError: 141887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3966.410324] Lustre: server umount lustre-MDT0001 complete [ 3977.082424] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 3979.392537] Key type lgssc unregistered [ 3979.589994] LNet: 145867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3979.594627] LNetError: 145867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3979.605644] LNet: Removed LNI 192.168.202.112@tcp [ 3980.342601] Key type .llcrypt unregistered [ 3980.344882] Key type ._llcrypt unregistered [ 4001.681941] Key type ._llcrypt registered [ 4001.683583] Key type .llcrypt registered [ 4001.760571] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4002.638271] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4002.793267] alg: No test for adler32 (adler32-zlib) [ 4003.787114] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 4004.035722] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4005.698426] Key type lgssc registered [ 4006.603810] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4012.747990] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4012.761110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4014.152227] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4017.702516] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4024.294455] LustreError: 147272:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4024.306391] LustreError: 147272:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4024.514043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4024.831418] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4028.145684] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4034.498882] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4038.983538] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4044.098661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4044.388191] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4049.293916] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4049.520982] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4055.781478] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4059.770519] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4065.238508] Lustre: *** cfs_fail_loc=159, val=0*** [ 4081.748913] Lustre: lustre-MDT0000: Client 924042ff-4397-4e2a-8311-f194e4ed3f13 (at 192.168.202.12@tcp) reconnecting [ 4085.730603] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4085.731609] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4085.732252] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4085.749738] Lustre: Skipped 1 previous similar message [ 4090.856891] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4090.859991] Lustre: Skipped 1 previous similar message [ 4091.385348] Lustre: server umount lustre-OST0000 complete [ 4095.972480] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4095.987741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4095.991767] Lustre: Skipped 2 previous similar messages [ 4100.300839] Lustre: server umount lustre-MDT0000 complete [ 4101.089962] LustreError: 149872:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4103.370256] LustreError: 147253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568823 with bad export cookie 7144595133840442809 [ 4103.376754] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4103.379993] LustreError: 147253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4103.640147] Lustre: server umount lustre-MDT0001 complete [ 4111.475211] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4113.805737] Key type lgssc unregistered [ 4114.018040] LNet: 150938:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4114.027156] LNetError: 150938:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4114.040926] LNet: Removed LNI 192.168.202.112@tcp [ 4114.723483] Key type .llcrypt unregistered [ 4114.727961] Key type ._llcrypt unregistered [ 4134.011272] Key type ._llcrypt registered [ 4134.013670] Key type .llcrypt registered [ 4134.087469] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4135.077607] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4135.092274] alg: No test for adler32 (adler32-zlib) [ 4136.103134] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 4136.258143] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4137.928964] Key type lgssc registered [ 4138.906823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4144.656259] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4144.670424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4145.974379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4149.257453] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4155.387453] LustreError: 152331:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4155.420918] LustreError: 152331:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4155.515279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4155.778734] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4158.999837] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4164.191245] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4168.578230] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4173.334805] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4173.586384] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4177.686078] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4180.789353] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4183.599737] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4190.028194] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4205.808635] Lustre: lustre-MDT0001: evicting ee452844-d2ac-4f85-b9f2-8156f82bbbbf (at 192.168.202.12@tcp) by administrative request [ 4205.817335] Lustre: Skipped 1 previous similar message [ 4232.161651] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4232.170678] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4232.180172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4234.213540] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4234.220484] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4237.982348] Lustre: server umount lustre-OST0000 complete [ 4242.919087] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4242.926403] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4242.955971] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4242.958916] Lustre: Skipped 1 previous similar message [ 4248.854473] Lustre: server umount lustre-MDT0000 complete [ 4249.574427] LustreError: 155450:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4249.584270] LustreError: 155450:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4251.489236] LustreError: 152311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786568971 with bad export cookie 7712478362563994257 [ 4251.494236] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4251.507086] LustreError: 152311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4251.821367] Lustre: server umount lustre-MDT0001 complete [ 4260.544972] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4262.622708] Key type lgssc unregistered [ 4262.856631] LNet: 156406:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4262.865159] LNetError: 156406:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4262.879321] LNet: Removed LNI 192.168.202.112@tcp [ 4263.479713] Key type .llcrypt unregistered [ 4263.483102] Key type ._llcrypt unregistered [ 4315.356394] Key type ._llcrypt registered [ 4315.358099] Key type .llcrypt registered [ 4315.450544] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4316.185133] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4316.262366] alg: No test for adler32 (adler32-zlib) [ 4317.217594] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 4317.424487] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4319.057070] Key type lgssc registered [ 4319.823687] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4324.819441] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4324.835982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4326.108492] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4328.818261] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4334.690284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4335.087526] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4339.141752] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4344.823384] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4348.991743] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4354.128275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4354.676698] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4359.443606] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4360.814560] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4366.164347] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4371.050551] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4381.672627] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4381.672879] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4381.680738] LustreError: Skipped 1 previous similar message [ 4381.691960] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4381.701533] Lustre: Skipped 1 previous similar message [ 4386.786605] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4386.798269] Lustre: Skipped 1 previous similar message [ 4391.392135] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4391.486343] Lustre: server umount lustre-OST0000 complete [ 4396.513670] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4396.525064] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4396.532922] Lustre: Skipped 1 previous similar message [ 4396.536262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4400.915107] Lustre: server umount lustre-MDT0000 complete [ 4402.146238] LustreError: 161183:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4402.167642] LustreError: 161183:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4403.245536] LustreError: 158851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569123 with bad export cookie 17766160316835528104 [ 4403.251081] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4403.252390] LustreError: 158851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4403.486941] Lustre: server umount lustre-MDT0001 complete [ 4409.671383] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4411.292841] Key type lgssc unregistered [ 4411.501321] LNet: 162415:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4411.504244] LNetError: 162415:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4411.517727] LNet: Removed LNI 192.168.202.112@tcp [ 4412.058418] Key type .llcrypt unregistered [ 4412.060244] Key type ._llcrypt unregistered [ 4427.142515] Key type ._llcrypt registered [ 4427.146217] Key type .llcrypt registered [ 4427.221812] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4427.923367] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4427.940938] alg: No test for adler32 (adler32-zlib) [ 4428.929148] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 4429.087829] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4430.720136] Key type lgssc registered [ 4431.376648] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4439.970910] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 17:12:39 (1786569159) [ 4450.629956] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 4457.615402] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4482.036190] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4482.082221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4483.258216] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4483.288840] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4483.347669] Lustre: lustre-MDT0000: new disk, initializing [ 4483.396727] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4483.414240] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4486.599988] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4493.757354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4493.834127] Lustre: 166651:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4493.862340] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4493.865195] Lustre: Skipped 1 previous similar message [ 4493.908517] Lustre: lustre-MDT0001: new disk, initializing [ 4493.960896] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4493.987402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4493.992139] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4496.488189] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4502.586495] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4506.418792] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4511.021624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4511.186664] Lustre: lustre-OST0000: new disk, initializing [ 4511.190335] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4511.196088] Lustre: 168619:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4511.260813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4515.371865] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4520.492578] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4520.499498] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4520.561165] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4521.494191] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4530.658652] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4530.665902] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4530.677288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4530.684553] Lustre: Skipped 1 previous similar message [ 4534.391670] Lustre: server umount lustre-OST0000 complete [ 4540.388019] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4540.405451] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4540.418186] Lustre: Skipped 1 previous similar message [ 4540.420509] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4546.017981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4546.026249] Lustre: Skipped 2 previous similar messages [ 4550.116714] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4550.125653] Lustre: Skipped 1 previous similar message [ 4551.136147] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4551.248810] Lustre: server umount lustre-MDT0000 complete [ 4553.648088] LustreError: 169772:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786569273 with bad export cookie 11947819578949547114 [ 4553.649284] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4553.655313] LustreError: 169772:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4553.906805] Lustre: server umount lustre-MDT0001 complete [ 4561.113654] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4563.057243] Key type lgssc unregistered [ 4563.341725] LNet: 170527:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4563.346106] LNetError: 170527:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4563.354419] LNet: Removed LNI 192.168.202.112@tcp [ 4564.045695] Key type .llcrypt unregistered [ 4564.047771] Key type ._llcrypt unregistered [ 4579.424300] Key type ._llcrypt registered [ 4579.425791] Key type .llcrypt registered [ 4579.490312] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4580.306346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4580.392220] alg: No test for adler32 (adler32-zlib) [ 4581.433447] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 4581.638802] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4583.304449] Key type lgssc registered [ 4584.062688] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4589.222633] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4589.239299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4590.562839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4593.355885] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4599.252890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4599.527239] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4602.780754] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4608.095819] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4611.786457] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4617.230821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4617.452047] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4621.565364] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4627.045169] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4635.885756] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4642.455037] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4642.473690] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4642.480834] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4642.485947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.494962] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4643.128509] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4643.145501] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4643.154507] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4643.160200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.171419] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4643.791682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.342440] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4644.345718] Lustre: Skipped 1 previous similar message [ 4644.355880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4644.363537] Lustre: Skipped 1 previous similar message [ 4644.366748] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4644.369835] Lustre: Skipped 1 previous similar message [ 4644.373218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.381077] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4644.384009] Lustre: Skipped 1 previous similar message [ 4644.960771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.503617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.238912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.873807] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4646.878750] Lustre: Skipped 3 previous similar messages [ 4646.890517] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4646.898805] Lustre: Skipped 3 previous similar messages [ 4646.903435] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4646.910221] Lustre: Skipped 3 previous similar messages [ 4646.912528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.923651] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4646.928338] Lustre: Skipped 3 previous similar messages [ 4647.696661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.286453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.006376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.649849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.312720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.873385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.490077] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4651.494308] Lustre: Skipped 6 previous similar messages [ 4651.519139] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4651.531352] Lustre: Skipped 6 previous similar messages [ 4651.537866] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4651.542289] Lustre: Skipped 6 previous similar messages [ 4651.544335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.547753] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4651.549408] Lustre: Skipped 6 previous similar messages [ 4652.125978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.808457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.396311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.014211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.666953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.300136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.823665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.399762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.053615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.706425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.203542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.798534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.406682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.924848] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4659.936915] Lustre: Skipped 13 previous similar messages [ 4659.946995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4659.956072] Lustre: Skipped 13 previous similar messages [ 4659.963618] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4659.966553] Lustre: Skipped 13 previous similar messages [ 4659.968936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.975792] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4659.984937] Lustre: Skipped 13 previous similar messages [ 4660.532225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.203862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.802802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.397395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.978698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.623466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.261184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.970974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.643872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.373336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.049961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.802377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.427960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.108958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.826847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.608576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.319278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.125703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.721342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.443701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.076307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.763440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.352057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.997267] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4676.002392] Lustre: Skipped 23 previous similar messages [ 4676.021800] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4676.027143] Lustre: Skipped 23 previous similar messages [ 4676.029410] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4676.032020] Lustre: Skipped 23 previous similar messages [ 4676.038794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.048960] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4676.053097] Lustre: Skipped 23 previous similar messages [ 4676.669950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.347762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.030956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.759745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.379799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.117532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.700382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.326909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.944317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.506136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.134175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.781128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.307803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.894239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.416452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.963967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.551962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.114226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.734247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.408322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.940642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.531655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.209048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.859454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.482949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.186115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.939207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.719878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.418343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.115502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.703847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.420383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.988443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.624664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.185690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.737512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.341337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.942329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.484649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.045970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.609920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.237709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.855944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.403844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.023231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.574139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.205176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.893606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.466108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.961275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.534850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.050278] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4708.058570] Lustre: Skipped 51 previous similar messages [ 4708.067480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4708.076336] Lustre: Skipped 51 previous similar messages [ 4708.083839] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4708.086867] Lustre: Skipped 51 previous similar messages [ 4708.090397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.100349] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4708.114996] Lustre: Skipped 51 previous similar messages [ 4708.625923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.287123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.816893] LustreError: 171941:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4709.821503] LustreError: 171941:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4709.826100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.361569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.895933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.475876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.987327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.472548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.978288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.522465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.047523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.718641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.280972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.842342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.348556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.934794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.489304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.049604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.582280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.169939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.701704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.207338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.828250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.345826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.877054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.490481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.969569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.518034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.062975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.609381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.153077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.715808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.324146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.939333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.653539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.344924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.928718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.523282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.125139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.706042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.327251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.922563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.485548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.077123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.612712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.247934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.851056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.584042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.222773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.812982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.373815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.970970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.619338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.202501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.841224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.461962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.024841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.597628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.126215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.646680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.247436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.780564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.364323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.958957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.666258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.290725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.916690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.466237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.068517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.657602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.282935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.868908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.370078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.967088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.577652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.161187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.692543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.356539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.023127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.608590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.203493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.844518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.443438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.039426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.604886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.122738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.643569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.173696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.666333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.188826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.734138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.250404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.739310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.253651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.726886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.248843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.805825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.437451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.015529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.602580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.246403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.832936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.477774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.029883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.596476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.205106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.807647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.376231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.944374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.426976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.924587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.375271] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4772.378088] Lustre: Skipped 111 previous similar messages [ 4772.388508] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4772.394091] Lustre: Skipped 111 previous similar messages [ 4772.396699] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4772.399875] Lustre: Skipped 111 previous similar messages [ 4772.402483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.408958] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4772.410969] Lustre: Skipped 111 previous similar messages [ 4772.895980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.365628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.843903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.345051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.832968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.421950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.948369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.473125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.976334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.549620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.078462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.634381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.136248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.662097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.192649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.632496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.163243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.763738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.255699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.777679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.292525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.808771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.403102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.977710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.452135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.032973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.565622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.078741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.592174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.083748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.647264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.168489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.675254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.111344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.532961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.939256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.436129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.904490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.418988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.991765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.480638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.020112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.517543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.077220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.544794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.018211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.590499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.046675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.549134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.094900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.560914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.059670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.592146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.059256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.552914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.084955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.602069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.039430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.451507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.922614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.387440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.834268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.309283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.781602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.230400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.713793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.183574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.612155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.007490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.431852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.809317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.209933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.609318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.030101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.483872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.990962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.464972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.903872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.347108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.835618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.298411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.839414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.360355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.879192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.427765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.061558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.598829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.169314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.688611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.307365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.917365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.481862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.083746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.688563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.219827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.793625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.399216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.999535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.539549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.102426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.629072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.338337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.948045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.613351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.180604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.726688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.297727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.925764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.504068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.118656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.681544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.308076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.802361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.337045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.835480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.430294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.978637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.549250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.173364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.764449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.265886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.812064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.364393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.972909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.541463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.120057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.671730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.301161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.858252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.386328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.969467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.536250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.081791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.618938] LustreError: 171941:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4842.624791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.628791] LustreError: 171941:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4843.175095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.784926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.317566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.826688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.354338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.876295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.398175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.869343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.351781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.932724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.399865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.964763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.493217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.070521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.631016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.218219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.768029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.408172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.912948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.471509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.972990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.532927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.019979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.574509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.059059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.623415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.172436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.747519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.278528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.746023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.222556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.678181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.186930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.795081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.286577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.744298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.189649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.684255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.191341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.754485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.334776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.869164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.414654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.978667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.536737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.032220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.536853] LustreError: 171941:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4867.544172] LustreError: 171941:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4867.547351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.020712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.514820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.042833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.569862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.074711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.678869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.282697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.871578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.454697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.049750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.619049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.177289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.763526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.333561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.858805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.378891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.989505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.504148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.077518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.682105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.247416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.852382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.375617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.875398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.410550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.950227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.414125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.866123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.381275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.846269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.431584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.985792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.539194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.109827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.636794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.085360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.608229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.160278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.647074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.167028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.741777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.359290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.887848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.404979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.919679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.409867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.892711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.348562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.864097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.354116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.859170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.377322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.915395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.511442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.115837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.710767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.315395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.920606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.426587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.886679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.417792] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4900.420247] Lustre: Skipped 241 previous similar messages [ 4900.429930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4900.435397] Lustre: Skipped 241 previous similar messages [ 4900.437805] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4900.440579] Lustre: Skipped 241 previous similar messages [ 4900.442916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.453933] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4900.458997] Lustre: Skipped 241 previous similar messages [ 4900.979106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.461153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.940174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.422536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.917775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.478458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.054380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.631755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.071150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.643110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.189492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.701378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.221332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.774412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.332039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.969398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.474552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.052072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.576907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.105314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.629739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.158883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.719758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.218221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.708753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.255395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.734456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.234447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.749919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.243278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.726565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.239475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.701171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.170120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.684959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.194255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.735666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.134171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.597876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.092947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.540820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.017397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.493991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.046176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.613070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.206894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.757534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.384391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.951694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.514762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.035333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.617578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.145502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.724903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.323133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.948330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.543663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.124358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.663718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.270295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.855111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.527831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.049751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.595664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.161838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.707726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.217190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.701712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.173525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.669056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.236964] LustreError: 171941:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4938.243899] LustreError: 171941:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4938.245494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.773118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.253868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.779647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.230751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.770555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.334024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.900027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.605598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.298302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.956384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.643285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.222940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.736860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.240352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.784417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.387462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.996171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.512714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.113657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.613501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.238180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.805537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.487799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.043818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.602319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.168566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.666785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.197034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.754815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.324831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.917409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.447856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.946681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.519419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.094358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.638329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.224304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.695985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.286936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.857501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.414451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.007079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.537252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.051919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.613301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.095396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.598274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.166518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.735429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.281067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.770114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.321506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.851648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.396428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.977604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.448624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.913780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.414792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.911772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.436900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.944419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.460736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.032033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.678134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.258124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.815106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.333605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.842245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.427791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.023312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.586665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.190174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.797773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.295751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.860636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.400159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.969798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.543843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.168275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.739788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.295546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.855969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.432622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.925904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.455577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.972988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.425437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.937827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.431347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.919614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.454167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.982493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.502555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.002910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.558350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.064311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.549030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.150088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.645623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.148893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.741079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.228599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.728935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.292339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.808433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.331283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.807390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.308095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.915076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.439680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.981658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.535681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.005925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.555480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.086226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.669975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.248556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.842162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.428989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.940043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.508818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.992440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.519275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.072990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.548161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.030187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.520800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.090184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.600827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.097538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.588186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.136332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.623529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.144488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.682124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.164548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.670582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.194443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.740687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.185289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.798791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.318357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.833880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.351916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.910456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.450059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.936710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.486642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.013687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.593032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.144313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.720904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.220655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.668478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.210591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.698728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.209180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.756369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.221898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.725139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.252080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.756180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.181317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.697104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.226888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.681412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.203781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.718761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.217238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.736776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.253559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.721397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.226970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.741945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.288664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.756093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.268650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.798948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.347112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.921037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.421376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.876119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.379810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.884711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.336498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.793681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.194778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.682267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.271270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.801047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.263615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.661855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.070893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.490810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.016731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.476632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.904330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.291150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.681688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.092772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.557761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.027642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.540328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.960494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.350247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.840370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.411892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.022811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.546357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.176477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.910610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.497684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.101944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.663667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.346867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.007493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.537952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.060495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.626499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.141936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.708388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.242412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.790787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.300593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.874903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.403907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.929702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.424935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.930487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.440940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.971706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.456500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.993975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.457849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.939514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.449268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.917502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.422057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.879645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.312304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.750231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.188842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.588379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.977381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.396249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.836699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.300901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.781448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.253504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.705671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.175907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.621782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.030302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.491956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.953643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.402442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.932996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.393517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.854251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.363777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.856497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.291615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.726767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.131801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.572890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.061061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.557796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.069033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.534680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.928803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.363751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.863922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.350819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.738610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.148449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.560950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.945209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.340584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.758321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.218974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.643977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.043320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.474903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.868300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.335345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.807061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.335266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.867316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.383511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.848444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.393755] LustreError: 171941:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5088.398467] LustreError: 171941:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5088.401867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.780485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.225650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.668574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.118432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.508377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.944397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.346706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.723244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.069564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.492198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.905848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.345195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.740733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.124781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.546573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.933823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.345049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.700474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.135580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.630936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.058375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.452937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.864683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.329339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.734620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.159160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.605095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.028717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.440471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.859475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.375910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.814661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.383267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.846546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.342260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.886880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.386196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.843721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.347430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.809612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.311387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.887846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.392308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.886936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.365193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.858391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.306054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.798174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.323118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.845243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.305094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.801938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.300584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.762888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.267294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.842564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.362204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.077980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.777769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.468851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.147323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.806641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.436539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.112615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.652750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.250615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.816759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.352736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.834774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.371127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.828149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.310096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.829807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.356157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.830352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.301484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.990919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.522947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.159846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.797994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.457669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.168659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.864683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.563559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.085249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.619021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.168561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.691861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.267910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.816064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.281803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.680133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.131091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.602329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.067061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.582258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.010031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.454926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.908438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.412839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.877933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.378600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.914776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.354602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.854687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.319260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.830674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.429565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.993960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.586818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.073892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.568433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.053790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.532618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.039953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.557518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.999589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.465947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.033992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.554410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.104094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.584841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.102464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.643426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.189566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.772465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.303859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.830751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.389136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.922116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.466916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.993871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.618521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.149344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.769431] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5156.778939] Lustre: Skipped 496 previous similar messages [ 5156.793940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5156.799110] Lustre: Skipped 496 previous similar messages [ 5156.803433] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5156.808148] Lustre: Skipped 496 previous similar messages [ 5156.820081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.833470] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5156.838362] Lustre: Skipped 496 previous similar messages [ 5157.421965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.104324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.692468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.350657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.079807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.727746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.247678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.872272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.450844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.074686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.596826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.138218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.680722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.300396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.967572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.659599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.264866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.766757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.341726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.843754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.385214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.967705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.520358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.103877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.652984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.211156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.712374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.226220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.692123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.166122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.677768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.237699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.714567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.216924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.862556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.466767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.100493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.644647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.409213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.096776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.678765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.304526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.845899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.432210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.602542] LustreError: 174410:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5508.608480] LustreError: 171941:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5732.842100] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 5732.844321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5732.850269] LustreError: Skipped 1 previous similar message [ 5732.868068] Lustre: Skipped 44 previous similar messages [ 5732.870404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5734.889680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5734.894131] Lustre: Skipped 1 previous similar message [ 5740.010573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5745.128488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5745.137841] Lustre: Skipped 2 previous similar messages [ 5747.168293] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5747.298303] Lustre: server umount lustre-OST0000 complete [ 5750.246346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5750.249513] Lustre: Skipped 1 previous similar message [ 5755.316730] Lustre: server umount lustre-MDT0000 complete [ 5755.365870] LustreError: 171922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5755.376393] LustreError: 171922:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5757.228143] LustreError: 171947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570477 with bad export cookie 12766778924721691989 [ 5757.229437] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5757.234718] LustreError: 171947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5757.428941] Lustre: server umount lustre-MDT0001 complete [ 5762.525979] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 5764.086142] Key type lgssc unregistered [ 5764.255589] LNet: 227056:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5764.259803] LNetError: 227056:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5764.271044] LNet: Removed LNI 192.168.202.112@tcp [ 5764.767364] Key type .llcrypt unregistered [ 5764.768844] Key type ._llcrypt unregistered [ 5777.358697] Key type ._llcrypt registered [ 5777.360036] Key type .llcrypt registered [ 5777.425734] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 5777.848378] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5777.924250] alg: No test for adler32 (adler32-zlib) [ 5778.847349] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 5778.950438] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 5780.560159] Key type lgssc registered [ 5781.152175] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5785.320487] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5785.328205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5786.578415] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5788.657670] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5792.983646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5793.188174] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5795.142547] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5798.863078] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5801.631764] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5804.765816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5804.920564] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5807.906408] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5811.053587] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 5811.771995] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5819.053289] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5819.953215] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5819.958676] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5819.966181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.978055] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5819.978086] LustreError: 228467:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5823.459586] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5823.466820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5827.114694] Lustre: server umount lustre-OST0000 complete [ 5839.984153] LustreError: 228467:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5856.245815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5856.421699] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5859.167192] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5862.517079] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5863.587655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.593768] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.594349] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5863.602216] Lustre: Skipped 1 previous similar message [ 5864.286660] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 5866.828258] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff932305883800.ost_server_uuid 50 [ 5867.518353] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff932305883800.ost_server_uuid in FULL state after 0 sec [ 5869.027210] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5869.029461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5869.041077] Lustre: Skipped 2 previous similar messages [ 5869.043648] Lustre: Skipped 2 previous similar messages [ 5874.145913] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5874.150028] Lustre: Skipped 1 previous similar message [ 5875.163377] Lustre: server umount lustre-OST0000 complete [ 5879.265363] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5879.267304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5879.273071] Lustre: Skipped 1 previous similar message [ 5879.278106] Lustre: Skipped 2 previous similar messages [ 5882.852636] Lustre: server umount lustre-MDT0000 complete [ 5884.386381] LustreError: 230945:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5884.393523] LustreError: 230945:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5884.586310] LustreError: 228429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570604 with bad export cookie 847681334968213106 [ 5884.588413] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5884.592857] LustreError: 228429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5884.731324] Lustre: server umount lustre-MDT0001 complete [ 5889.522078] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 5890.980859] Key type lgssc unregistered [ 5891.150590] LNet: 233655:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5891.157644] LNetError: 233655:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5891.169366] LNet: Removed LNI 192.168.202.112@tcp [ 5891.696575] Key type .llcrypt unregistered [ 5891.698553] Key type ._llcrypt unregistered [ 5906.345748] Key type ._llcrypt registered [ 5906.346834] Key type .llcrypt registered [ 5906.392277] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 5906.887682] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5906.903453] alg: No test for adler32 (adler32-zlib) [ 5907.791783] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 5907.896235] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 5909.496210] Key type lgssc registered [ 5910.001679] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5914.683902] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5914.699465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5915.950603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5917.949712] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5921.688577] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5921.847460] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5923.453748] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5924.584330] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5927.064892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5927.223369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5929.442256] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5932.575755] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5932.645662] Lustre: lustre-OST0001: new disk, initializing [ 5932.648123] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5932.650547] Lustre: 237679:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5932.696940] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5934.881511] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5938.717696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5938.728157] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5938.770864] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5940.786335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 5946.114380] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5955.296324] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5962.730430] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5962.735167] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5962.743300] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5966.305261] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5966.308949] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5966.312128] Lustre: Skipped 1 previous similar message [ 5968.681904] Lustre: server umount lustre-MDT0000 complete [ 5970.164669] LustreError: 235355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570690 with bad export cookie 14588793920787900522 [ 5970.167995] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5970.171505] LustreError: 235355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5970.313570] Lustre: server umount lustre-MDT0001 complete [ 5981.066961] Lustre: server umount lustre-OST0000 complete [ 5992.035694] Lustre: server umount lustre-OST0001 complete [ 5999.817260] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6003.519602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6003.672567] LustreError: 240786:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6003.680651] LustreError: 240786:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6003.700420] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6005.251463] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6008.698059] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6010.383737] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6011.487261] Lustre: 241928:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6014.027522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6014.165939] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6014.168969] Lustre: Skipped 1 previous similar message [ 6016.432675] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6018.278686] LustreError: 242281:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6020.177615] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6022.959947] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6028.780741] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 6031.878497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6033.102264] Lustre: 243796:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6039.011548] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6039.013155] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6039.017527] Lustre: Skipped 4 previous similar messages [ 6039.020443] Lustre: Skipped 5 previous similar messages [ 6043.815060] Lustre: server umount mylustre-MDT0000 complete [ 6044.131073] LustreError: 240787:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6044.137945] LustreError: 240787:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6045.270101] LustreError: 240761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570765 with bad export cookie 14588793920787903301 [ 6045.272262] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6045.274740] LustreError: 240761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6045.458113] Lustre: server umount mylustre-MDT0001 complete [ 6057.478508] Lustre: server umount mylustre-OST0000 complete [ 6068.193124] Lustre: server umount mylustre-OST0001 complete [ 6077.388741] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6081.529885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6081.710513] LustreError: 246100:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6081.737309] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6081.740326] Lustre: Skipped 1 previous similar message [ 6083.375813] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6086.795050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6088.639151] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6089.806529] Lustre: 247240:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6092.511421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6095.134808] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6096.806504] LustreError: 248164:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6098.762800] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6101.482585] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6106.088382] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 6106.089364] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 6110.395513] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6112.074559] Lustre: 249108:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6116.322384] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6116.328383] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6116.332583] Lustre: Skipped 3 previous similar messages [ 6116.339972] Lustre: Skipped 3 previous similar messages [ 6120.748592] Lustre: server umount tfs-MDT0000 complete [ 6121.443097] LustreError: 246097:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6121.449909] LustreError: 246097:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6122.283677] LustreError: 246842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570842 with bad export cookie 14588793920787904967 [ 6122.287814] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6122.289309] LustreError: 246842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6122.423783] Lustre: server umount tfs-MDT0001 complete [ 6134.797626] Lustre: server umount tfs-OST0000 complete [ 6146.670211] Lustre: server umount tfs-OST0001 complete [ 6154.818576] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6158.711146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6158.872340] LustreError: 251356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6158.892445] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6158.894437] Lustre: Skipped 3 previous similar messages [ 6160.483917] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6164.269979] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6166.376685] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6167.704802] Lustre: 252499:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6170.798664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6173.479928] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6177.216147] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6179.744635] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6183.403875] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 6183.405368] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 6188.891187] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6190.435627] Lustre: 254369:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6193.349258] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 17:41:52 (1786570912) [ 6205.409722] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6205.414619] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6205.420233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6211.568768] Lustre: server umount lustre-MDT0000 complete [ 6213.160362] LustreError: 252877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786570933 with bad export cookie 14588793920787906633 [ 6213.163786] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6213.165361] LustreError: 252877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6214.113353] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6214.113811] LustreError: 251353:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6214.114837] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6214.114841] Lustre: Skipped 4 previous similar messages [ 6214.117930] Lustre: Skipped 4 previous similar messages [ 6214.125021] LustreError: 251353:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6219.326345] Lustre: server umount lustre-MDT0001 complete [ 6235.104131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6235.152417] Lustre: server umount lustre-OST0000 complete [ 6242.904402] Lustre: server umount lustre-OST0001 complete [ 6245.296942] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 6249.160823] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6263.024951] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 6263.024951] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6263.024951] [ 6263.032183] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6263.121814] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6263.136051] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6263.173292] Lustre: lustre-MDT0000: new disk, initializing [ 6263.198131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6263.200451] Lustre: Skipped 3 previous similar messages [ 6263.206754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6264.653650] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6269.365500] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 6269.365500] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6269.365500] [ 6269.374933] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6269.428647] Lustre: 257547:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6269.460829] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6269.463030] Lustre: Skipped 1 previous similar message [ 6269.500118] Lustre: lustre-MDT0001: new disk, initializing [ 6269.572377] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6269.582705] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6271.330316] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6275.926506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6276.053202] Lustre: lustre-OST0000: new disk, initializing [ 6276.055504] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6276.060641] Lustre: 259128:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6277.387589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6277.394276] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6277.453286] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6278.518055] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6283.705561] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6283.790985] Lustre: lustre-OST0001: new disk, initializing [ 6283.796274] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6283.799606] Lustre: 259999:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6285.393110] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6285.397719] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6285.457230] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6286.311468] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6291.394272] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 17:43:30 (1786571010) [ 6295.811883] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 17:43:35 (1786571015) [ 6308.322840] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6308.322880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6308.327546] LustreError: Skipped 1 previous similar message [ 6308.328153] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6308.332260] Lustre: Skipped 1 previous similar message [ 6308.333739] Lustre: Skipped 3 previous similar messages [ 6313.614516] Lustre: server umount lustre-OST0000 complete [ 6315.489244] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6316.518169] LustreError: 259311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6316.526299] LustreError: 259311:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6321.386451] Lustre: server umount lustre-MDT0000 complete [ 6322.713295] LustreError: 257539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571042 with bad export cookie 14588793920787908180 [ 6322.718431] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6322.719353] LustreError: 257539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6336.992131] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6337.089404] Lustre: server umount lustre-MDT0001 complete [ 6341.369423] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 6411.174661] Lustre: server umount lustre-OST0001 complete [ 6413.510385] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 6420.108446] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6433.446881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6433.533158] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6433.546064] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6433.584376] Lustre: lustre-MDT0000: new disk, initializing [ 6433.608749] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6433.612458] Lustre: Skipped 3 previous similar messages [ 6433.619249] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6435.196020] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6439.934310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6439.999261] Lustre: 264771:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6440.118360] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6441.647941] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6445.616654] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6447.658665] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6450.183353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6450.291956] Lustre: lustre-OST0000: new disk, initializing [ 6450.293431] Lustre: Skipped 1 previous similar message [ 6450.295256] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6450.297302] Lustre: Skipped 2 previous similar messages [ 6450.299614] Lustre: 266741:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6451.392077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6451.397641] Lustre: Skipped 1 previous similar message [ 6451.400445] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6451.447662] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6452.531643] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6456.612191] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6461.921783] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6461.922352] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6461.925631] LustreError: Skipped 1 previous similar message [ 6461.930085] Lustre: Skipped 4 previous similar messages [ 6461.931262] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6461.936702] Lustre: Skipped 11 previous similar messages [ 6465.450121] Lustre: server umount lustre-OST0000 complete [ 6481.376120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6481.441063] Lustre: server umount lustre-MDT0000 complete [ 6482.401711] LustreError: 267751:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6482.407342] LustreError: 267751:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 6482.707283] LustreError: 264764:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571202 with bad export cookie 14588793920787909657 [ 6482.710583] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6482.711273] LustreError: 264764:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6482.849251] Lustre: server umount lustre-MDT0001 complete [ 6486.478365] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 6487.540387] Key type lgssc unregistered [ 6487.664434] LNet: 268507:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6487.667465] LNetError: 268507:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6487.679280] LNet: Removed LNI 192.168.202.112@tcp [ 6488.001136] Key type .llcrypt unregistered [ 6488.002472] Key type ._llcrypt unregistered [ 6497.977275] Key type ._llcrypt registered [ 6497.978160] Key type .llcrypt registered [ 6498.018379] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6498.329595] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6498.345541] alg: No test for adler32 (adler32-zlib) [ 6499.219142] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 6499.327039] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 6500.912161] Key type lgssc registered [ 6501.305640] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6503.994208] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6504.000710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6505.149362] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6506.510643] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6509.330571] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6509.462696] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6510.873839] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6513.526484] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6515.462536] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6517.842051] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6517.953374] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6519.963278] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6522.400358] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6522.766593] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6524.386321] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6524.387103] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6524.390467] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6525.518861] Lustre: server umount lustre-OST0000 complete [ 6526.772616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6526.777190] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6526.777745] Lustre: Skipped 1 previous similar message [ 6532.064957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6532.067798] Lustre: Skipped 2 previous similar messages [ 6532.892126] Lustre: server umount lustre-MDT0000 complete [ 6534.136084] LustreError: 269928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571254 with bad export cookie 622788635442125468 [ 6534.138933] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6534.139955] LustreError: 269928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6534.251042] Lustre: server umount lustre-MDT0001 complete [ 6537.849589] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 6538.905777] Key type lgssc unregistered [ 6539.035477] LNet: 273521:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6539.038046] LNetError: 273521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6539.045312] LNet: Removed LNI 192.168.202.112@tcp [ 6539.345170] Key type .llcrypt unregistered [ 6539.346170] Key type ._llcrypt unregistered [ 6546.275699] Key type ._llcrypt registered [ 6546.276827] Key type .llcrypt registered [ 6546.320631] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6546.756631] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6546.772281] alg: No test for adler32 (adler32-zlib) [ 6547.636047] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 6547.719397] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 6549.304159] Key type lgssc registered [ 6549.682933] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6552.872209] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6552.879868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6568.288352] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6573.408268] LustreError: 274692:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.112@tcp: failed processing log, type 4: rc = -110 [ 6600.032194] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6602.082191] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6604.741678] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6605.519239] Lustre: server umount lustre-OST0000 complete [ 6611.182526] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 6612.214223] Key type lgssc unregistered [ 6612.338308] LNet: 276323:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6612.341598] LNetError: 276323:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6612.349296] LNet: Removed LNI 192.168.202.112@tcp [ 6612.657088] Key type .llcrypt unregistered [ 6612.658071] Key type ._llcrypt unregistered [ 6626.973913] Key type ._llcrypt registered [ 6626.975407] Key type .llcrypt registered [ 6627.022729] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6627.456203] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6627.479057] alg: No test for adler32 (adler32-zlib) [ 6628.370448] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 6628.479606] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 6630.064236] Key type lgssc registered [ 6630.445988] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6635.135678] loop0: detected capacity change from 0 to 204800000 [ 6635.201854] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6635.206934] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6635.327165] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6635.679933] loop0: detected capacity change from 0 to 204800000 [ 6635.735373] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6635.739371] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6635.858365] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6636.218549] loop0: detected capacity change from 0 to 204800000 [ 6636.283907] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6636.287546] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6636.327562] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6636.714220] loop0: detected capacity change from 0 to 204800000 [ 6636.767733] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6637.151350] loop0: detected capacity change from 0 to 204800000 [ 6637.164741] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6638.542663] loop0: detected capacity change from 0 to 204800000 [ 6638.557299] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6639.967704] loop0: detected capacity change from 0 to 204800000 [ 6639.978489] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6641.262349] loop0: detected capacity change from 0 to 204800000 [ 6641.273408] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6642.603874] loop0: detected capacity change from 0 to 204800000 [ 6642.635060] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6642.644997] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6642.680694] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6642.685639] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6642.689698] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6642.694081] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6642.698788] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6642.703053] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6642.706842] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6642.711234] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6642.715645] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6642.721229] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6642.725382] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6642.731595] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6642.736114] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6642.741183] Lustre: 279711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6645.388785] Lustre: server umount MGS complete [ 6645.696677] loop0: detected capacity change from 0 to 204800000 [ 6645.721525] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6645.877339] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6645.893065] Lustre: lustre-MDT0000: Aborting client recovery [ 6646.228545] loop1: detected capacity change from 0 to 204800000 [ 6646.262319] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6646.307055] Lustre: 280141:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6646.312413] Lustre: 280141:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6646.320909] Lustre: 280141:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6646.325106] Lustre: 280141:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6646.329265] Lustre: 280141:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6646.335059] Lustre: 280141:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6646.340524] Lustre: 280141:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6646.349070] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6646.416567] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6646.444937] Lustre: lustre-MDT0001: Aborting client recovery [ 6646.775482] loop2: detected capacity change from 0 to 204800000 [ 6646.807877] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6646.841347] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6646.846635] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6646.850873] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6646.855848] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6646.859630] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6646.863487] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6646.868094] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6646.914522] Lustre: Found index 0 for lustre-OST0000, updating log [ 6646.924929] Lustre: 280241:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6646.947173] Lustre: lustre-OST0000: Aborting recovery [ 6647.278949] loop3: detected capacity change from 0 to 204800000 [ 6647.307788] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6647.348554] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6647.354388] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6647.360309] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6647.365649] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6647.372778] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6647.378740] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6647.384996] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6647.423300] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6647.426408] Lustre: Skipped 1 previous similar message [ 6652.903549] LustreError: 280085:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 6655.970330] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6655.971142] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6655.971713] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6655.976215] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6655.981274] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6655.993342] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6658.061260] LustreError: 280051:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 275us: rc = -2 [ 6671.328632] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6671.329320] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6671.332906] Lustre: Skipped 1 previous similar message [ 6675.551722] Lustre: server umount lustre-MDT0000 complete [ 6675.891522] LustreError: 280034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571395 with bad export cookie 18095510839739174238 [ 6675.893386] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6675.896271] LustreError: 280034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6676.448503] LustreError: 280049:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6676.448617] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6676.448974] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6676.448978] Lustre: Skipped 3 previous similar messages [ 6676.454327] LustreError: 280049:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6676.457498] Lustre: Skipped 3 previous similar messages [ 6680.480614] LustreError: 280050:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6680.480625] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6680.480630] Lustre: Skipped 1 previous similar message [ 6680.485078] LustreError: 280050:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6685.602987] LustreError: 280048:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6685.605410] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6685.618078] LustreError: 280048:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6685.621677] Lustre: Skipped 1 previous similar message [ 6690.272200] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6690.479773] Lustre: server umount lustre-MDT0001 complete [ 6697.276319] Lustre: server umount lustre-OST0000 complete [ 6712.288157] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6712.385328] Lustre: server umount lustre-OST0001 complete [ 6718.867713] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:50:37 (1786571437) [ 6738.184136] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 6745.562927] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6767.240229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6767.331086] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6767.351305] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6767.399689] Lustre: lustre-MDT0000: new disk, initializing [ 6767.446033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6767.466766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6769.402087] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6774.411346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6774.462490] Lustre: 284397:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6774.497249] LustreError: 284408:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6774.500356] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6774.505302] Lustre: Skipped 1 previous similar message [ 6774.545469] Lustre: lustre-MDT0001: new disk, initializing [ 6774.587554] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6774.602071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6774.606723] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6776.226491] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6780.970826] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6783.798562] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6786.874683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6786.988345] Lustre: lustre-OST0000: new disk, initializing [ 6786.991105] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6786.993408] Lustre: 286369:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6786.996563] Lustre: 286369:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 6787.034945] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6789.495667] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6793.743688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6793.750165] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6793.813583] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6794.517414] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6797.297415] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6801.901575] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6809.076288] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6850.020843] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6850.021485] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6850.022031] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6850.024638] LustreError: Skipped 1 previous similar message [ 6850.029287] Lustre: Skipped 1 previous similar message [ 6852.696737] Lustre: server umount lustre-OST0000 complete [ 6855.137339] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6855.145340] Lustre: Skipped 1 previous similar message [ 6860.258645] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6860.267392] Lustre: Skipped 4 previous similar messages [ 6863.296199] Lustre: server umount lustre-MDT0000 complete [ 6865.294441] LustreError: 284388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571585 with bad export cookie 18095510839739176947 [ 6865.306889] LustreError: 284388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6865.309348] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6865.527758] Lustre: server umount lustre-MDT0001 complete [ 6870.835154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6872.904461] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6875.836967] Lustre: server umount MGS complete [ 6882.353907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6882.609609] LustreError: 290227:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6882.624744] LustreError: 290240:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6882.663209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6885.388550] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6890.668034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6893.131313] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6896.939326] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6899.739727] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6903.623578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6907.178613] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6910.983313] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6934.497113] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6934.500102] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6934.504563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6935.522460] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6939.542560] Lustre: server umount lustre-OST0000 complete [ 6942.176954] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6945.764274] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6945.768864] Lustre: Skipped 2 previous similar messages [ 6947.224539] Lustre: server umount lustre-MDT0000 complete [ 6949.238524] LustreError: 292952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571669 with bad export cookie 18095510839739178760 [ 6949.240221] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6949.243307] LustreError: 292952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6949.387297] Lustre: server umount lustre-MDT0001 complete [ 6954.087185] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 6955.268669] Key type lgssc unregistered [ 6955.408504] LNet: 294170:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6955.411817] LNetError: 294170:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6955.418384] LNet: Removed LNI 192.168.202.112@tcp [ 6955.811314] Key type .llcrypt unregistered [ 6955.812732] Key type ._llcrypt unregistered [ 6970.086725] Key type ._llcrypt registered [ 6970.087929] Key type .llcrypt registered [ 6970.127520] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 6977.058903] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6977.511973] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6977.523611] alg: No test for adler32 (adler32-zlib) [ 6978.413567] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 6978.532485] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 6980.128178] Key type lgssc registered [ 6980.554842] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6994.155794] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6994.163960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6995.255914] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6995.268229] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6995.304845] Lustre: lustre-MDT0000: new disk, initializing [ 6995.334302] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6995.345540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6997.019054] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7002.180368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7002.235401] Lustre: 298102:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7002.262755] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7002.264511] Lustre: Skipped 1 previous similar message [ 7002.298423] Lustre: lustre-MDT0001: new disk, initializing [ 7002.340881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7002.355095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7002.360355] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7004.281886] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7008.598483] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7010.816550] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7013.572621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7013.706599] Lustre: lustre-OST0000: new disk, initializing [ 7013.709832] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7013.713338] Lustre: 300069:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7013.760214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7016.134524] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7017.029342] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7017.033766] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7017.066905] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7020.069374] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7023.671927] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7025.249557] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7032.781416] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7037.477815] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 7078.372733] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7078.373673] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7078.379779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7078.389800] Lustre: Skipped 1 previous similar message [ 7082.411963] Lustre: server umount lustre-OST0000 complete [ 7084.514312] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7084.516749] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7084.521439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7084.523445] Lustre: Skipped 1 previous similar message [ 7088.609119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7088.611194] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7090.290426] Lustre: server umount lustre-MDT0000 complete [ 7091.726061] LustreError: 300067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571811 with bad export cookie 18096631150087783857 [ 7091.727248] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7091.730359] LustreError: 300067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7091.873362] Lustre: server umount lustre-MDT0001 complete [ 7095.190690] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7096.784328] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7098.427063] Lustre: server umount MGS complete [ 7102.494026] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7102.734611] LustreError: 303901:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7102.764326] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7104.317544] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7107.539947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7107.697409] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7109.292043] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7111.980989] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7113.867515] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7116.214946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7116.335608] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7118.455567] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7120.975383] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7151.010672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7151.012461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7151.016317] Lustre: Skipped 1 previous similar message [ 7151.020142] Lustre: Skipped 2 previous similar messages [ 7156.193241] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7156.195522] Lustre: Skipped 1 previous similar message [ 7156.944726] Lustre: server umount lustre-OST0000 complete [ 7158.753143] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7158.756251] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7164.307944] Lustre: server umount lustre-MDT0000 complete [ 7165.640289] LustreError: 303868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786571885 with bad export cookie 18096631150087785796 [ 7165.643683] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7165.644190] LustreError: 303868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7165.763675] Lustre: server umount lustre-MDT0001 complete [ 7170.855864] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 7171.927404] Key type lgssc unregistered [ 7172.060302] LNet: 307852:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7172.063272] LNetError: 307852:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7172.073324] LNet: Removed LNI 192.168.202.112@tcp [ 7172.369357] Key type .llcrypt unregistered [ 7172.370842] Key type ._llcrypt unregistered [ 7179.040193] Key type ._llcrypt registered [ 7179.041309] Key type .llcrypt registered [ 7179.076266] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 7185.099467] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7185.434500] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7185.490617] alg: No test for adler32 (adler32-zlib) [ 7186.355025] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 7186.450256] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 7188.032154] Key type lgssc registered [ 7188.423814] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7196.603968] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7201.816583] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7201.823584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7202.932319] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7202.949548] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7202.992190] Lustre: lustre-MDT0000: new disk, initializing [ 7203.028121] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7203.036079] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7204.506791] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7208.805398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7208.922658] Lustre: lustre-OST0000: new disk, initializing [ 7208.924617] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7208.926631] Lustre: Skipped 1 previous similar message [ 7208.929443] Lustre: 311878:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7208.970632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7209.013172] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7209.020335] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7209.031793] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 7211.108851] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7424.281515] Lustre: server umount lustre-MDT0000 complete [ 7425.643295] LustreError: 311066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572145 with bad export cookie 16347871799422252617 [ 7425.648254] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7435.309060] Lustre: server umount lustre-OST0000 complete [ 7443.976816] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 7447.359056] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7460.049613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7460.129416] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7460.143925] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7460.180475] Lustre: lustre-MDT0000: new disk, initializing [ 7460.208087] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7460.215931] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7461.692492] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7465.816875] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7465.860423] Lustre: 316120:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7465.884713] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7465.886923] Lustre: Skipped 1 previous similar message [ 7465.924867] Lustre: lustre-MDT0001: new disk, initializing [ 7465.962500] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7465.974908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7465.978756] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7467.290012] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7471.040512] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7473.040165] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7475.502330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7475.613713] Lustre: lustre-OST0000: new disk, initializing [ 7475.616322] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7475.620353] Lustre: 318088:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7475.666170] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7477.782627] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7481.590529] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7483.098370] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7484.919895] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7484.923497] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7484.953812] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7485.235456] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 7486.771404] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7486.850034] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7490.018449] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7490.018859] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7490.020935] LustreError: Skipped 1 previous similar message [ 7490.021762] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7490.025453] Lustre: Skipped 1 previous similar message [ 7493.651496] Lustre: server umount lustre-OST0000 complete [ 7495.137770] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7495.138455] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7495.141303] Lustre: Skipped 1 previous similar message [ 7495.143103] Lustre: Skipped 1 previous similar message [ 7500.257303] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7500.260673] Lustre: Skipped 2 previous similar messages [ 7501.015222] Lustre: server umount lustre-MDT0000 complete [ 7502.192109] LustreError: 316115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572222 with bad export cookie 16347871799425193408 [ 7502.194827] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7502.195284] LustreError: 316115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7502.299767] Lustre: server umount lustre-MDT0001 complete [ 7503.435255] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7504.017326] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 18:03:43 (1786572223) [ 7506.354058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7506.531742] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7507.980415] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7510.816157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7512.313349] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7514.734361] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7516.656444] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7519.510120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7522.196586] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7525.882956] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7530.325295] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7530.476943] Lustre: lustre-OST0001: new disk, initializing [ 7530.479553] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7530.487728] Lustre: 323243:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7530.560238] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7530.563573] Lustre: Skipped 2 previous similar messages [ 7535.237511] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7539.239463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7539.247041] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7539.272468] LustreError: 323830:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7540.326462] LustreError: 323835:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7540.339638] LustreError: 323835:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 7541.346468] LustreError: 323830:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7541.874877] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7542.034388] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7543.396452] LustreError: 323830:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7543.410758] LustreError: 323830:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 7545.970830] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff932307aea000.ost_server_uuid 50 [ 7547.012848] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff932307aea000.ost_server_uuid in FULL state after 0 sec [ 7547.558386] LustreError: 323835:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7547.569280] LustreError: 323835:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [ 7550.635842] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff932307aea000.ost_server_uuid 50 [ 7551.810408] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff932307aea000.ost_server_uuid in FULL state after 0 sec [ 7558.119665] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7564.775562] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7564.777702] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7564.783408] Lustre: Skipped 1 previous similar message [ 7569.451982] Lustre: lustre-OST0001: Not available for connect from 192.168.202.12@tcp (stopping) [ 7569.456507] Lustre: Skipped 1 previous similar message [ 7570.715282] Lustre: server umount lustre-OST0001 complete [ 7574.479183] Lustre: server umount lustre-OST0000 complete [ 7577.573038] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7577.581626] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7577.592165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7577.594649] Lustre: Skipped 2 previous similar messages [ 7586.329473] Lustre: server umount lustre-MDT0000 complete [ 7588.497843] LustreError: 320391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572308 with bad export cookie 16347871799425194276 [ 7588.501686] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7588.504152] LustreError: 320391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7598.144777] Lustre: server umount lustre-MDT0001 complete [ 7603.314635] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 7604.718403] Key type lgssc unregistered [ 7604.909732] LNet: 325927:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7604.913273] LNetError: 325927:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7604.925437] LNet: Removed LNI 192.168.202.112@tcp [ 7605.391336] Key type .llcrypt unregistered [ 7605.394233] Key type ._llcrypt unregistered [ 7619.172646] Key type ._llcrypt registered [ 7619.173989] Key type .llcrypt registered [ 7619.236290] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7619.733679] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7619.811974] alg: No test for adler32 (adler32-zlib) [ 7620.747735] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 7620.861704] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 7622.480117] Key type lgssc registered [ 7623.101921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7627.317767] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7627.333497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7628.597508] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7630.904855] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7634.600678] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7637.987155] LustreError: 327327:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7638.010047] LustreError: 327327:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7638.267476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7638.502480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7640.544679] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7644.906871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7645.077524] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7647.799604] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7648.165772] LustreError: 329407:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7650.942527] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7653.289632] LustreError: 329004:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7653.300261] LustreError: 329004:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7653.308569] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 7654.099925] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7654.214331] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7656.848867] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7659.516304] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7660.310546] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7664.657605] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 7664.775764] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7751.651074] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7751.653874] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7751.662674] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7751.665698] Lustre: Skipped 1 previous similar message [ 7753.994437] Lustre: server umount lustre-OST0001 complete [ 7756.769260] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7756.769349] LustreError: 329007:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7756.771914] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7756.774184] Lustre: Skipped 2 previous similar messages [ 7756.779270] LustreError: 329007:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7761.891593] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7761.892205] LustreError: 329407:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7761.894810] Lustre: Skipped 2 previous similar messages [ 7761.900260] LustreError: 329407:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7762.266916] Lustre: server umount lustre-OST0000 complete [ 7765.880069] Lustre: server umount lustre-MDT0000 complete [ 7766.496894] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7766.500343] LustreError: Skipped 1 previous similar message [ 7766.502435] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7767.314356] LustreError: 328256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572487 with bad export cookie 15108852819311664668 [ 7767.316834] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7767.319181] LustreError: 328256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7767.446375] Lustre: server umount lustre-MDT0001 complete [ 7771.351272] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 7772.505674] Key type lgssc unregistered [ 7772.644498] LNet: 332263:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7772.647969] LNetError: 332263:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7772.655755] LNet: Removed LNI 192.168.202.112@tcp [ 7773.003925] Key type .llcrypt unregistered [ 7773.005223] Key type ._llcrypt unregistered [ 7792.238733] Key type ._llcrypt registered [ 7792.239786] Key type .llcrypt registered [ 7792.275773] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7792.742332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7792.753092] alg: No test for adler32 (adler32-zlib) [ 7793.609744] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 7793.704771] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 7795.288142] Key type lgssc registered [ 7795.677340] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7798.883960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7798.890919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7800.060444] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7801.516798] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7804.844452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7805.016819] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7806.528488] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7807.776886] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7810.375773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7810.506228] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7811.556484] LustreError: 336015:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7811.560957] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 7811.562554] LustreError: 336015:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7812.676723] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7815.650673] LustreError: 336015:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7815.656436] LustreError: 336015:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7815.863370] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7815.985649] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7818.185337] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7821.292746] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7822.620317] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7828.761993] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7835.616826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7835.619494] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7835.624890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7836.645362] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7836.645679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7836.652818] Lustre: Skipped 2 previous similar messages [ 7836.655594] Lustre: Skipped 2 previous similar messages [ 7838.689571] Lustre: server umount lustre-MDT0000 complete [ 7840.065269] LustreError: 337532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572560 with bad export cookie 1396427808638113449 [ 7840.069609] LustreError: 337532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7840.069748] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7840.192338] Lustre: server umount lustre-MDT0001 complete [ 7851.981540] Lustre: server umount lustre-OST0000 complete [ 7863.579212] Lustre: server umount lustre-OST0001 complete [ 7869.907368] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7873.877074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7874.046369] LustreError: 339785:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7874.083121] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7875.784169] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7879.137130] LustreError: 339786:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7879.353453] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7881.389033] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7882.802988] Lustre: 340925:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7885.506374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7885.678681] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7885.681368] Lustre: Skipped 1 previous similar message [ 7888.117310] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7889.769675] LustreError: 341282:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7892.263339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7895.015829] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7899.631841] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 7899.632548] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7904.009307] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7905.333696] Lustre: 342795:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7909.858910] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7909.860809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7909.862840] Lustre: Skipped 2 previous similar messages [ 7909.864641] Lustre: Skipped 1 previous similar message [ 7912.620875] Lustre: server umount lustre-MDT0000 complete [ 7914.093838] LustreError: 339768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572634 with bad export cookie 1396427808638115136 [ 7914.097174] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7914.099386] LustreError: 339768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7914.230754] Lustre: server umount lustre-MDT0001 complete [ 7925.072281] Lustre: server umount lustre-OST0000 complete [ 7936.814988] Lustre: server umount lustre-OST0001 complete [ 7938.333433] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 7938.930709] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 7939.644734] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 18:10:58 (1786572658) [ 7948.583543] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7951.899935] loop0: detected capacity change from 0 to 1099511627776 [ 7952.077240] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 7952.081884] *** VALIDATE xfs *** [ 7952.089593] XFS (loop0): Mounting V5 Filesystem [ 7952.100190] XFS (loop0): Ending clean mount [ 7952.611479] loop1: detected capacity change from 0 to 18691697672192 [ 7974.705228] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 7978.075979] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7990.077095] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7990.172219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7990.183142] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7990.222105] Lustre: lustre-MDT0000: new disk, initializing [ 7990.244408] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7990.248470] Lustre: Skipped 1 previous similar message [ 7990.256495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7991.650836] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7995.907388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7995.951662] Lustre: 347906:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7995.983533] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7995.986703] Lustre: Skipped 1 previous similar message [ 7996.026315] Lustre: lustre-MDT0001: new disk, initializing [ 7996.079617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7996.083550] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7997.490198] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8001.392810] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8003.315819] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8005.655564] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8005.758350] Lustre: lustre-OST0000: new disk, initializing [ 8005.760582] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8005.763891] Lustre: 349875:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8007.294259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8007.297939] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8007.340394] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8007.785889] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8011.705249] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8013.194447] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8013.271591] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8014.764209] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8014.844751] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8017.891942] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8017.893259] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8017.895088] LustreError: Skipped 1 previous similar message [ 8017.895878] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8017.899882] Lustre: Skipped 2 previous similar messages [ 8017.901023] Lustre: Skipped 2 previous similar messages [ 8021.666872] Lustre: server umount lustre-OST0000 complete [ 8027.105083] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8027.108386] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8027.113399] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8029.342141] Lustre: server umount lustre-MDT0000 complete [ 8030.785099] LustreError: 347899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572750 with bad export cookie 1396427808638116669 [ 8030.787399] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8030.790093] LustreError: 347899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8030.909181] Lustre: server umount lustre-MDT0001 complete [ 8033.752081] XFS (loop0): Unmounting Filesystem [ 8035.382573] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 18:12:34 (1786572754) [ 8037.769165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8037.961032] LustreError: 352429:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8037.995374] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8037.997829] Lustre: Skipped 2 previous similar messages [ 8039.464708] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8042.621208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8044.300214] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8046.956891] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8049.055751] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8051.787390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8054.377362] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8057.511607] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8073.697250] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8073.699842] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8073.703165] Lustre: Skipped 1 previous similar message [ 8073.705336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8073.707493] Lustre: Skipped 2 previous similar messages [ 8077.458673] Lustre: server umount lustre-OST0000 complete [ 8078.816959] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8078.819166] LustreError: Skipped 1 previous similar message [ 8090.081217] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8090.084097] Lustre: Skipped 8 previous similar messages [ 8093.152137] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8093.218418] Lustre: server umount lustre-MDT0000 complete [ 8094.597383] LustreError: 352410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786572814 with bad export cookie 1396427808638117551 [ 8094.600698] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8094.602111] LustreError: 352410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8094.731722] Lustre: server umount lustre-MDT0001 complete [ 8098.706213] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 8099.847525] Key type lgssc unregistered [ 8099.975864] LNet: 356082:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8099.978946] LNetError: 356082:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8099.992698] LNet: Removed LNI 192.168.202.112@tcp [ 8100.329220] Key type .llcrypt unregistered [ 8100.330676] Key type ._llcrypt unregistered [ 8111.320953] Key type ._llcrypt registered [ 8111.322256] Key type .llcrypt registered [ 8111.366174] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 8111.829717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8111.840275] alg: No test for adler32 (adler32-zlib) [ 8112.705572] Lustre: Lustre: Build Version: 2.17.57_2_g9cc1fed [ 8112.798816] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 8114.384156] Key type lgssc registered [ 8114.809081] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8117.717373] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8117.725454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8118.870816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8120.349665] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8123.555208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8123.697373] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8125.277952] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8127.963517] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8130.062559] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8132.458228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8132.575052] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8134.581360] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8137.126101] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8152.612461] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.12@tcp (stopping) [ 8154.082201] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8154.087637] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8154.091406] Lustre: Skipped 1 previous similar message [ 8156.369265] Lustre: server umount lustre-MDT0001 complete [ 8157.725616] LustreError: 357620:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.12@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8157.731680] LustreError: 357620:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8159.203428] LustreError: 357621:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8159.210140] LustreError: 357621:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8160.342238] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8160.403516] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8160.406196] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8160.428927] Lustre: 357614:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8160.558607] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8162.062631] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8165.867929] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8167.912708] LustreError: 361370:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8167.916396] LustreError: 361370:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8167.919549] LustreError: 357617:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8167.940426] LustreError: 361376:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.112@tcp: cfg command failed: rc = -17 [ 8167.946043] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 8167.946043] [ 8464.934828] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.12@tcp (stopping) [ 8467.937382] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8467.938130] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8467.939380] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8467.945290] Lustre: Skipped 2 previous similar messages [ 8470.224468] Lustre: server umount lustre-MDT0001 complete [ 8473.057720] LustreError: 358977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8473.065145] LustreError: 358977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8474.094324] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8474.149210] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8474.151753] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8474.170703] Lustre: 357614:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8474.306512] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8475.822806] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8479.719499] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8483.818529] LustreError: 373853:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8483.821689] LustreError: 373853:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8483.824882] LustreError: 357617:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8483.827651] LustreError: 357617:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8483.851158] LustreError: 373867:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 8483.854475] LustreError: 373867:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 8483.856804] LustreError: 373867:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.112@tcp: cfg command failed: rc = -17 [ 8483.860483] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.112@tcp [ 8483.860483] [ 8781.794652] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8781.795262] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8781.801724] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8781.805069] Lustre: Skipped 3 previous similar messages [ 8784.085894] Lustre: server umount lustre-MDT0001 complete [ 8786.914356] LustreError: 358977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8786.920596] LustreError: 358977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8787.496199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8787.525025] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8787.526735] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8787.538358] Lustre: 357615:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8787.665068] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 8788.997918] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8793.061740] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8793.574424] LustreError: 386255:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8793.577532] LustreError: 386255:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8793.580163] LustreError: 357617:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8793.583383] LustreError: 357617:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8793.602260] LustreError: 386258:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 8793.607353] LustreError: 386258:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.112@tcp: cfg command failed: rc = -114 [ 8793.610036] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 8793.610036] [ 9110.507733] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9110.517333] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9110.524403] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9110.526893] Lustre: Skipped 3 previous similar messages [ 9110.535915] Lustre: Skipped 3 previous similar messages [ 9113.315520] Lustre: server umount lustre-MDT0000 complete [ 9115.618896] LustreError: 357625:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9115.624334] LustreError: 357625:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9116.040311] LustreError: 357607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786573836 with bad export cookie 8377392704268991714 [ 9116.050211] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9120.743383] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9122.553684] Lustre: server umount lustre-MDT0001 complete [ 9124.821749] Lustre: server umount lustre-OST0000 complete [ 9129.404065] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 9133.870181] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 9149.672378] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:31:08 (1786573868) [ 9152.740626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9152.845564] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9152.863911] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9152.922216] Lustre: lustre-MDT0000: new disk, initializing [ 9152.974217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9152.982364] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9155.026893] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9160.862414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9160.926332] Lustre: 400669:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9160.967539] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9160.974329] Lustre: Skipped 1 previous similar message [ 9161.038221] Lustre: lustre-MDT0001: new disk, initializing [ 9161.114269] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9161.146507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9161.158516] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9163.181459] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9168.240231] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9170.615299] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9173.690347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9173.854099] Lustre: lustre-OST0000: new disk, initializing [ 9173.857685] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9173.861927] Lustre: 402637:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9173.913110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9176.702423] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9177.283151] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9177.287642] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9177.318576] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9181.481061] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9186.785335] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9186.788316] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9186.803553] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9186.806271] Lustre: Skipped 2 previous similar messages [ 9191.137984] Lustre: server umount lustre-MDT0000 complete [ 9192.520407] LustreError: 401464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786573912 with bad export cookie 8377392704268993653 [ 9192.523555] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9192.524916] LustreError: 401464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9192.527157] LustreError: 401477:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 9192.527272] LustreError: 401477:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000a1f2bc00 x1873357394253184/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 9192.671861] Lustre: server umount lustre-MDT0001 complete [ 9204.596596] Lustre: server umount lustre-OST0000 complete [ 9218.762789] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 9223.313161] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 9236.483017] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 8948 sec ========== 18:32:35 (1786573955) [ 9237.133404] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:32:36 (1786573956) === [ 9238.644435] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:32:37 (1786573957) === [ 9258.370641] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 9259.947257] Key type lgssc unregistered [ 9260.115604] LNet: 408993:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9260.119687] LNetError: 408993:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9260.131496] LNet: Removed LNI 192.168.202.112@tcp [ 9260.541351] Key type .llcrypt unregistered [ 9260.543190] Key type ._llcrypt unregistered