[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 359143227 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001008] APIC: Switch to symmetric I/O mode setup [ 0.001985] x2apic enabled [ 0.002003] Switched APIC routing to physical x2apic. [ 0.003005] kvm-guest: setup PV IPIs [ 0.004962] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.005000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.005007] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.006003] pid_max: default: 32768 minimum: 301 [ 0.007062] LSM: Security Framework initializing [ 0.008020] Yama: becoming mindful. [ 0.009014] SELinux: Initializing. [ 0.009602] *** VALIDATE selinux *** [ 0.015472] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.019290] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.020070] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.022002] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023045] *** VALIDATE tmpfs *** [ 0.024094] *** VALIDATE proc *** [ 0.024766] *** VALIDATE cgroup *** [ 0.025003] *** VALIDATE cgroup2 *** [ 0.026134] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.027081] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.028002] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.029014] Spectre V2 : User space: Vulnerable [ 0.029731] Speculative Store Bypass: Vulnerable [ 0.031808] debug: unmapping init [mem 0xffffffffbb659000-0xffffffffbb660fff] [ 0.033775] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.034415] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.035008] ... version: 2 [ 0.035643] ... bit width: 48 [ 0.036005] ... generic registers: 4 [ 0.036692] ... value mask: 0000ffffffffffff [ 0.037005] ... max period: 00007fffffffffff [ 0.037856] ... fixed-purpose events: 3 [ 0.038005] ... event mask: 000000070000000f [ 0.039036] rcu: Hierarchical SRCU implementation. [ 0.040815] smp: Bringing up secondary CPUs ... [ 0.041328] x86: Booting SMP configuration: [ 0.042008] .... node #0, CPUs: #1 #2 #3 [ 0.044309] smp: Brought up 1 node, 4 CPUs [ 0.045675] smpboot: Max logical packages: 1 [ 0.046005] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.117017] node 0 deferred pages initialised in 70ms [ 0.119089] devtmpfs: initialized [ 0.119796] x86/mm: Memory block size: 128MB [ 0.120968] gcov: version magic: 0x41383552 [ 0.121439] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.122049] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.123120] pinctrl core: initialized pinctrl subsystem [ 0.124081] [ 0.124372] ************************************************************* [ 0.125004] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.126004] ** ** [ 0.127004] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.128004] ** ** [ 0.129004] ** This means that this kernel is built to expose internal ** [ 0.130005] ** IOMMU data structures, which may compromise security on ** [ 0.131004] ** your system. ** [ 0.132004] ** ** [ 0.133004] ** If you see this message and you are not debugging the ** [ 0.134004] ** kernel, report this immediately to your vendor! ** [ 0.135004] ** ** [ 0.136004] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.137005] ************************************************************* [ 0.138343] NET: Registered protocol family 16 [ 0.139287] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.140021] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.141021] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.142230] cpuidle: using governor menu [ 0.143322] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.144259] PCI: Using configuration type 1 for base access [ 0.145073] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.150058] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.151009] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.152112] cryptd: max_cpu_qlen set to 1000 [ 0.153738] ACPI: Added _OSI(Module Device) [ 0.155008] ACPI: Added _OSI(Processor Device) [ 0.155900] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.156006] ACPI: Added _OSI(Processor Aggregator Device) [ 0.158813] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.163209] ACPI: Interpreter enabled [ 0.163976] ACPI: PM: (supports S0 S3 S4 S5) [ 0.165006] ACPI: Using IOAPIC for interrupt routing [ 0.166051] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.169351] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.178088] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.181016] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.183007] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.184031] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.188223] acpiphp: Slot [2] registered [ 0.189064] acpiphp: Slot [5] registered [ 0.189874] acpiphp: Slot [6] registered [ 0.191059] acpiphp: Slot [7] registered [ 0.191754] acpiphp: Slot [8] registered [ 0.192054] acpiphp: Slot [9] registered [ 0.192908] acpiphp: Slot [10] registered [ 0.194050] acpiphp: Slot [3] registered [ 0.194862] acpiphp: Slot [4] registered [ 0.195049] acpiphp: Slot [11] registered [ 0.195861] acpiphp: Slot [12] registered [ 0.197045] acpiphp: Slot [13] registered [ 0.197960] acpiphp: Slot [14] registered [ 0.199043] acpiphp: Slot [15] registered [ 0.199889] acpiphp: Slot [16] registered [ 0.201050] acpiphp: Slot [17] registered [ 0.201869] acpiphp: Slot [18] registered [ 0.203048] acpiphp: Slot [19] registered [ 0.203865] acpiphp: Slot [20] registered [ 0.204039] acpiphp: Slot [21] registered [ 0.204881] acpiphp: Slot [22] registered [ 0.206048] acpiphp: Slot [23] registered [ 0.206869] acpiphp: Slot [24] registered [ 0.208038] acpiphp: Slot [25] registered [ 0.208876] acpiphp: Slot [26] registered [ 0.209061] acpiphp: Slot [27] registered [ 0.209863] acpiphp: Slot [28] registered [ 0.211040] acpiphp: Slot [29] registered [ 0.211905] acpiphp: Slot [30] registered [ 0.212040] acpiphp: Slot [31] registered [ 0.212886] PCI host bridge to bus 0000:00 [ 0.214009] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.215015] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.217008] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.219007] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.220007] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.222016] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.223125] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.225665] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.227381] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.233456] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.237034] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.239008] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.240006] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.242007] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.243471] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.245594] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.248026] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.251713] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.255979] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.266010] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.270008] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.273262] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.278008] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.282007] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.296008] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.303298] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.308006] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.313007] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.329010] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.335434] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.339008] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.342814] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.351847] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.358659] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.362823] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.365943] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.377883] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.384135] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.387930] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.391905] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.404983] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.411000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.415006] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.418914] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.430842] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.438252] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.439271] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.441217] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.442164] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.443110] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.446026] iommu: Default domain type: Passthrough [ 0.448342] SCSI subsystem initialized [ 0.450097] ACPI: bus type USB registered [ 0.452095] usbcore: registered new interface driver usbfs [ 0.454045] usbcore: registered new interface driver hub [ 0.456037] usbcore: registered new device driver usb [ 0.457123] pps_core: LinuxPPS API ver. 1 registered [ 0.459008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.462020] PTP clock support registered [ 0.463083] EDAC MC: Ver: 3.0.0 [ 0.464097] PCI: Using ACPI for IRQ routing [ 0.465660] NetLabel: Initializing [ 0.467007] NetLabel: domain hash size = 128 [ 0.467877] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.469042] NetLabel: unlabeled traffic allowed by default [ 0.470067] vgaarb: loaded [ 0.471231] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.473008] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.478186] clocksource: Switched to clocksource kvm-clock [ 0.557946] VFS: Disk quotas dquot_6.6.0 [ 0.558926] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.560311] *** VALIDATE ramfs *** [ 0.561104] *** VALIDATE hugetlbfs *** [ 0.562027] pnp: PnP ACPI init [ 0.563655] pnp: PnP ACPI: found 6 devices [ 0.575362] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.577334] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.578720] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.580043] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.581470] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.582821] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.584389] NET: Registered protocol family 2 [ 0.585846] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.588729] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.590827] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.594700] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.596794] TCP: Hash tables configured (established 65536 bind 65536) [ 0.598714] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.600514] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.602314] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.604101] NET: Registered protocol family 1 [ 0.605541] RPC: Registered named UNIX socket transport module. [ 0.606927] RPC: Registered udp transport module. [ 0.607923] RPC: Registered tcp transport module. [ 0.608847] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.610056] NET: Registered protocol family 44 [ 0.610841] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.612271] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.613659] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.615105] PCI: CLS 0 bytes, default 64 [ 0.616553] Unpacking initramfs... [ 1.728591] debug: unmapping init [mem 0xffff8c7c3cc54000-0xffff8c7c3ffbffff] [ 1.731715] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.734208] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 1.736356] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.151587] Initialise system trusted keyrings [ 2.152923] Key type blacklist registered [ 2.154402] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.161841] zbud: loaded [ 2.164158] *** VALIDATE nfs *** [ 2.165266] *** VALIDATE nfs4 *** [ 2.166629] pstore: using deflate compression [ 2.168742] Platform Keyring initialized [ 2.230299] NET: Registered protocol family 38 [ 2.231867] Key type asymmetric registered [ 2.233094] Asymmetric key parser 'x509' registered [ 2.233996] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.235908] io scheduler mq-deadline registered [ 2.236940] io scheduler kyber registered [ 2.237917] io scheduler bfq registered [ 2.238868] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.240317] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.241904] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.243402] ACPI: Power Button [PWRF] [ 2.246404] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.250879] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.260445] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.265204] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.277710] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.303802] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.330400] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.334331] Non-volatile memory driver v1.3 [ 2.335726] Linux agpgart interface v0.103 [ 2.359642] virtio_blk virtio1: [vda] 145888 512-byte logical blocks (74.7 MB/71.2 MiB) [ 2.362381] vda: detected capacity change from 0 to 74694656 [ 2.371662] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.373495] vdb: detected capacity change from 0 to 1073741824 [ 2.382902] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.384725] vdc: detected capacity change from 0 to 2621440000 [ 2.395533] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.397485] vdd: detected capacity change from 0 to 2621440000 [ 2.408532] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.410114] vde: detected capacity change from 0 to 4294967296 [ 2.419391] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.421168] vdf: detected capacity change from 0 to 4294967296 [ 2.425513] libphy: Fixed MDIO Bus: probed [ 2.428651] usbcore: registered new interface driver usbserial_generic [ 2.430636] usbserial: USB Serial support registered for generic [ 2.432467] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.435919] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.437520] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.439479] mousedev: PS/2 mouse device common for all mice [ 2.441631] rtc_cmos 00:05: RTC can wake from S4 [ 2.443552] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.444443] rtc_cmos 00:05: registered as rtc0 [ 2.448042] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.448261] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.450067] intel_pstate: CPU model not supported [ 2.453331] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.457394] hid: raw HID events driver (C) Jiri Kosina [ 2.458572] usbcore: registered new interface driver usbhid [ 2.460390] usbhid: USB HID core driver [ 2.461665] drop_monitor: Initializing network drop monitor service [ 2.463926] Initializing XFRM netlink socket [ 2.465539] NET: Registered protocol family 10 [ 2.467336] Segment Routing with IPv6 [ 2.468345] NET: Registered protocol family 17 [ 2.469905] mpls_gso: MPLS GSO support [ 2.475395] RAS: Correctable Errors collector initialized. [ 2.477445] AVX version of gcm_enc/dec engaged. [ 2.478918] AES CTR mode by8 optimization enabled [ 2.525826] sched_clock: Marking stable (2525799823, 0)->(3099359890, -573560067) [ 2.528350] registered taskstats version 1 [ 2.529636] Loading compiled-in X.509 certificates [ 2.531118] zswap: loaded using pool lzo/zbud [ 2.545667] Key type big_key registered [ 2.553105] Key type encrypted registered [ 2.554201] ima: No TPM chip found, activating TPM-bypass! [ 2.555326] ima: Allocated hash algorithm: sha1 [ 2.556216] ima: No architecture policies found [ 2.557067] evm: Initialising EVM extended attributes: [ 2.557971] evm: security.selinux [ 2.558662] evm: security.ima [ 2.559224] evm: security.capability [ 2.559889] evm: HMAC attrs: 0x1 [ 2.561346] rtc_cmos 00:05: setting system clock to 2026-07-29 09:05:33 UTC (1785315933) [ 2.564948] debug: unmapping init [mem 0xffffffffbc603000-0xffffffffbc7fffff] [ 2.566692] debug: unmapping init [mem 0xffffffffbb382000-0xffffffffbb658fff] [ 2.573079] Write protecting the kernel read-only data: 28672k [ 2.574745] debug: unmapping init [mem 0xffffffffb9a03000-0xffffffffb9bfffff] [ 2.576098] debug: unmapping init [mem 0xffffffffba314000-0xffffffffba3fffff] [ 2.595068] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 2.599929] systemd[1]: Detected virtualization kvm. [ 2.601388] systemd[1]: Detected architecture x86-64. [ 2.602573] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 2.624828] systemd[1]: No hostname configured. [ 2.626025] systemd[1]: Set hostname to . [ 2.627597] random: systemd: uninitialized urandom read (16 bytes read) [ 2.629060] systemd[1]: Initializing machine ID from random generator. [ 2.649776] random: ln: uninitialized urandom read (6 bytes read) [ 2.716093] random: systemd: uninitialized urandom read (16 bytes read) [ 2.717741] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 2.720827] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 2.723586] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Paths. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.090653] device-mapper: uevent: version 1.0.3 [ 3.091913] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ 3.605595] random: fast init done [ 3.614952] virtio_net virtio0 ens2: renamed from eth0 [ 3.620584] scsi host0: ata_piix [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 3.638516] scsi host1: ata_piix [ 3.639500] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 3.641028] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 Starting dracut initqueue hook... [ 7.885970] dracut-initqueue[590]: RTNETLINK answers: File exists [ 8.063402] random: crng init done [ 8.064857] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 8.402802] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 9.255401] printk: systemd: 26 output lines suppressed due to ratelimiting [ 9.398700] SELinux: Disabled at runtime. [ 9.446407] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 9.450946] systemd[1]: Detected virtualization kvm. [ 9.451887] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 9.761169] systemd[1]: initrd-switch-root.service: Succeeded. [ 9.763193] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 9.765620] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 9.767930] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 9.769781] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 9.773562] systemd[1]: Starting Journal Service... Starting Journal Service... [ 9.776244] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Stopped target Initrd File Systems. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on udev Control Socket. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-sshd\x2dkeygen.slice. Activating swap /dev/disk/by-label/SWAP... Mounting POSIX Message Queue File System... Mounting Huge Pages File System... Starting udev Coldplug all Devices... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ 9.885083] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 10.097474] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 10.392797] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 10.465273] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 10.533829] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 10.571485] EDAC sbridge: Ver: 1.1.2 [ 11.687058] Key type dns_resolver registered [ 11.945351] NFS: Registering the id_resolver key type [ 11.946556] Key type id_resolver registered [ 11.947609] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg629-server login: [ 32.096951] libcfs: loading out-of-tree module taints kernel. [ 32.114173] Key type ._llcrypt registered [ 32.115738] Key type .llcrypt registered [ 32.147400] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 39.421057] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 40.007895] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 40.013673] alg: No test for adler32 (adler32-zlib) [ 40.995286] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 41.294761] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 42.904193] Key type lgssc registered [ 43.481410] Lustre: Echo OBD driver; http://www.lustre.org/ [ 49.818058] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 62.614655] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 66.786091] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 66.794910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 67.885244] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 67.900849] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 67.935718] Lustre: lustre-MDT0000: new disk, initializing [ 67.956725] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 67.962825] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 69.211211] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 73.983688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 74.020258] Lustre: 6478:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 74.029782] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 74.032502] Lustre: Skipped 1 previous similar message [ 74.073565] Lustre: lustre-MDT0001: new disk, initializing [ 74.092369] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 74.101068] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 74.105173] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 75.286548] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 77.411404] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 80.220905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 80.299247] Lustre: lustre-OST0000: new disk, initializing [ 80.300857] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 80.302928] Lustre: 8414:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 80.316158] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 82.062296] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 83.241382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 83.243520] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 83.259774] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 87.317563] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 87.371108] Lustre: lustre-OST0001: new disk, initializing [ 87.372735] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 87.374658] Lustre: 9486:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 87.392269] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 89.307384] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 91.628375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 91.631942] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 91.645673] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 95.139278] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 98.753670] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 100.607953] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing check_logdir /tmp/testlogs/ [ 101.960050] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing yml_node [ 103.339872] Lustre: DEBUG MARKER: Client: 2.17.56.1 [ 104.094301] Lustre: DEBUG MARKER: MDS: 2.17.56.1 [ 104.862482] Lustre: DEBUG MARKER: OSS: 2.17.56.1 [ 105.375125] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Jul 29 05:07:15 EDT 2026 [ 110.572184] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 111.047353] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 125.408615] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 125.410740] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 125.414283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 127.456772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 127.456990] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 127.458479] Lustre: Skipped 1 previous similar message [ 127.463277] Lustre: Skipped 1 previous similar message [ 128.862961] Lustre: server umount lustre-MDT0000 complete [ 130.033665] LustreError: 6471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316060 with bad export cookie 3699869961642260195 [ 130.035261] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 130.038700] LustreError: 6471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 132.576813] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 132.577157] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 132.580282] Lustre: Skipped 2 previous similar messages [ 132.581832] Lustre: Skipped 2 previous similar messages [ 135.584483] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 135.586292] Lustre: Skipped 2 previous similar messages [ 139.680654] LustreError: 6486:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 139.680689] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 139.684388] LustreError: 6486:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 139.685805] Lustre: Skipped 1 previous similar message [ 144.352087] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 144.419750] Lustre: server umount lustre-MDT0001 complete [ 151.742330] Lustre: server umount lustre-OST0000 complete [ 167.392106] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 167.447225] Lustre: server umount lustre-OST0001 complete [ 169.543141] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 172.594625] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 184.615051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 184.703652] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 184.715567] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 184.751439] Lustre: lustre-MDT0000: new disk, initializing [ 184.772573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 184.779132] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 186.137476] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 190.263584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 190.306237] Lustre: 15027:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 190.410029] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 191.739123] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 195.454322] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 197.224869] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 199.439198] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 199.539866] Lustre: lustre-OST0000: new disk, initializing [ 199.541336] Lustre: Skipped 1 previous similar message [ 199.543059] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 199.545058] Lustre: Skipped 2 previous similar messages [ 199.546782] Lustre: 16994:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 199.577132] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 199.579647] Lustre: Skipped 1 previous similar message [ 201.030583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 201.034261] Lustre: Skipped 1 previous similar message [ 201.036283] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 201.088041] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 201.520829] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 205.200811] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 206.645282] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 206.716332] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 208.092595] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 208.172297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 211.427342] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 211.427614] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 211.429604] LustreError: Skipped 1 previous similar message [ 211.434648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 211.436908] Lustre: Skipped 1 previous similar message [ 214.927720] Lustre: server umount lustre-OST0000 complete [ 216.545138] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 216.550652] Lustre: Skipped 1 previous similar message [ 221.152888] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 222.364299] Lustre: server umount lustre-MDT0000 complete [ 223.556278] LustreError: 15022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316154 with bad export cookie 3699869961642261714 [ 223.559323] LustreError: 15022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 223.560199] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 223.683464] Lustre: server umount lustre-MDT0001 complete [ 225.360641] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 05:09:15 (1785316155) [ 227.577929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 227.727552] LustreError: 19256:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 227.750930] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 229.072477] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 231.840395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 233.403077] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 235.950515] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 237.821880] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 240.136935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 242.049290] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 244.284147] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 256.793936] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 262.624941] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 262.627825] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 262.632791] Lustre: Skipped 1 previous similar message [ 262.635177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 262.638230] Lustre: Skipped 3 previous similar messages [ 265.883748] Lustre: server umount lustre-MDT0000 complete [ 267.116358] LustreError: 21805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316198 with bad export cookie 3699869961642262596 [ 267.119371] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 267.124419] LustreError: 21805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 267.239603] Lustre: server umount lustre-MDT0001 complete [ 278.742600] Lustre: server umount lustre-OST0000 complete [ 285.682878] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 289.209934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 289.336406] LustreError: 23655:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 289.350202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 289.351969] Lustre: Skipped 2 previous similar messages [ 290.748682] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 293.563567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 295.095095] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 296.088523] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 298.285487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 300.093467] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 302.986219] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 303.057594] Lustre: lustre-OST0001: new disk, initializing [ 303.060471] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 303.064478] Lustre: 25969:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 304.840344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 304.844385] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 304.894509] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 305.154895] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 309.710460] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 312.328283] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 312.330368] Lustre: Skipped 1 previous similar message [ 314.337390] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 314.340299] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 314.344275] Lustre: Skipped 2 previous similar messages [ 314.346360] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 314.349665] Lustre: Skipped 3 previous similar messages [ 320.161128] Lustre: server umount lustre-MDT0000 complete [ 320.482140] LustreError: 23655:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 320.487289] LustreError: 23655:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 321.369106] LustreError: 23636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316252 with bad export cookie 3699869961642263443 [ 321.372604] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 321.373928] LustreError: 23636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 321.478115] Lustre: server umount lustre-MDT0001 complete [ 332.805932] Lustre: server umount lustre-OST0000 complete [ 343.186658] Lustre: server umount lustre-OST0001 complete [ 348.029456] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 351.146330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 351.258531] LustreError: 28930:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 352.512690] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 355.131527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 355.252785] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 355.254373] Lustre: Skipped 4 previous similar messages [ 356.479251] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 357.348885] Lustre: 30079:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 359.336470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 361.239186] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 361.509222] LustreError: 30432:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 363.957827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 365.935560] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 368.943997] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 370.021281] Lustre: 31954:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 375.268142] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 375.271812] Lustre: Skipped 5 previous similar messages [ 377.622516] Lustre: server umount lustre-MDT0000 complete [ 378.814208] LustreError: 29681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316309 with bad export cookie 3699869961642264675 [ 378.816850] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 378.819125] LustreError: 29681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 378.921223] Lustre: server umount lustre-MDT0001 complete [ 390.590745] Lustre: server umount lustre-OST0000 complete [ 402.009207] Lustre: server umount lustre-OST0001 complete [ 404.882506] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 05:12:15 (1785316335) [ 413.829460] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 417.091928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 417.224038] LustreError: 34748:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 418.592387] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 421.454779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 422.958605] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 423.962663] Lustre: 35897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 426.204329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 428.361917] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 431.176209] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 433.279611] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 436.345371] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 437.477427] Lustre: 37773:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 442.336896] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 442.339350] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 442.343955] Lustre: Skipped 1 previous similar message [ 442.346402] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 442.348697] Lustre: Skipped 8 previous similar messages [ 444.887074] Lustre: server umount lustre-MDT0000 complete [ 446.154116] LustreError: 34729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316377 with bad export cookie 3699869961642265767 [ 446.158452] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 446.159357] LustreError: 34729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 446.282713] Lustre: server umount lustre-MDT0001 complete [ 457.156197] Lustre: server umount lustre-OST0000 complete [ 468.638958] Lustre: server umount lustre-OST0001 complete [ 471.368759] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 05:13:21 (1785316401) [ 476.514637] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 479.686912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.825938] LustreError: 39908:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 479.832494] LustreError: 39908:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 481.114889] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 483.674652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 483.803437] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 483.805350] Lustre: Skipped 7 previous similar messages [ 484.994456] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 485.923214] Lustre: 41057:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 487.999826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 489.927963] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 492.481634] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 494.415061] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 497.530740] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 498.976123] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 502.043939] Lustre: server umount lustre-MDT0000 complete [ 503.302494] LustreError: 40657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316434 with bad export cookie 3699869961642266866 [ 503.307576] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 503.308538] LustreError: 40657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 509.482749] Lustre: server umount lustre-MDT0001 complete [ 512.390333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 513.440103] LustreError: 43791:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 513.443659] LustreError: 43791:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff8c7c87f30380 x1872039480935680/t0(0) o250->MGC192.168.206.129@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1785316444 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_00.0' uid:0 gid:0 projid:4294967295 [ 513.451358] LustreError: 43791:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 514.019210] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x335894209ad35528 [ 514.022222] Lustre: MGC192.168.206.129@tcp: Connection restored to 0@lo (at 0@lo) [ 515.557985] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 518.453367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 520.039277] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 522.359777] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 532.066572] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 9 sec [ 533.928834] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 533.995746] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 534.512331] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 539.107298] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 541.911799] Lustre: server umount lustre-MDT0000 complete [ 543.106993] LustreError: 39891:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316474 with bad export cookie 3699869961642267944 [ 543.110371] LustreError: 39891:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 543.110602] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 543.225292] Lustre: server umount lustre-MDT0001 complete [ 554.690504] Lustre: server umount lustre-OST0000 complete [ 565.148798] Lustre: server umount lustre-OST0001 complete [ 568.065535] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 05:14:58 (1785316498) [ 573.469342] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 577.057860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 577.200142] LustreError: 47531:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 577.207243] LustreError: 47531:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 578.619714] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 581.506303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 583.061123] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 584.008122] Lustre: 48684:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 584.011529] Lustre: 48684:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 586.116926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 588.161116] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 591.019057] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 593.173886] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 596.374392] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 597.904292] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 602.080850] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 602.082984] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 602.086325] Lustre: Skipped 11 previous similar messages [ 602.087893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 602.089481] Lustre: Skipped 13 previous similar messages [ 607.453348] Lustre: server umount lustre-MDT0000 complete [ 608.645222] LustreError: 50563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316539 with bad export cookie 3699869961642268770 [ 608.649085] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 608.650147] LustreError: 50563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 608.759030] Lustre: server umount lustre-MDT0001 complete [ 620.035223] Lustre: server umount lustre-OST0000 complete [ 631.509451] Lustre: server umount lustre-OST0001 complete [ 634.691274] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 05:16:05 (1785316565) [ 635.503270] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 636.081761] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 05:16:06 (1785316566) [ 639.147824] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 641.977103] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 649.040138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 649.126083] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 649.160605] Lustre: lustre-MDT0000: new disk, initializing [ 649.184853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 650.474606] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 654.157664] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 656.377713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 656.487232] Lustre: 54745:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 658.242407] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 658.250756] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 658.465659] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 662.212749] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 662.970244] Lustre: DEBUG MARKER: create test files [ 673.762236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 679.497899] Lustre: server umount lustre-OST0000 complete [ 680.877696] Lustre: server umount lustre-MDT0000 complete [ 684.616726] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 685.655789] Key type lgssc unregistered [ 685.787581] LNet: 56354:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 685.789915] LNetError: 56354:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 685.798301] LNet: Removed LNI 192.168.206.129@tcp [ 686.149117] Key type .llcrypt unregistered [ 686.150981] Key type ._llcrypt unregistered [ 696.828968] Key type ._llcrypt registered [ 696.829971] Key type .llcrypt registered [ 696.870094] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 697.318788] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 697.343111] alg: No test for adler32 (adler32-zlib) [ 698.206935] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 698.308308] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 699.888108] Key type lgssc registered [ 700.261035] Lustre: Echo OBD driver; http://www.lustre.org/ [ 702.894140] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 702.902040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 704.046810] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 705.395294] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 707.906820] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 710.256799] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 710.365407] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 712.329422] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 713.447417] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 713.474602] Lustre: 59063:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 713.477510] Lustre: 59063:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 713.480914] Lustre: 59063:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 713.485091] Lustre: 59063:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 713.488035] Lustre: 59063:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 713.491286] Lustre: 59063:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 714.790658] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 717.835304] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 719.544812] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 723.937778] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 723.937902] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 723.941818] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 727.632459] Lustre: server umount lustre-OST0000 complete [ 728.990203] Lustre: server umount lustre-MDT0000 complete [ 732.724270] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 733.732772] Key type lgssc unregistered [ 733.857338] LNet: 60673:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 733.859538] LNetError: 60673:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 733.867328] LNet: Removed LNI 192.168.206.129@tcp [ 734.165542] Key type .llcrypt unregistered [ 734.166508] Key type ._llcrypt unregistered [ 745.334234] Key type ._llcrypt registered [ 745.335218] Key type .llcrypt registered [ 745.373989] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 745.809981] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 745.819825] alg: No test for adler32 (adler32-zlib) [ 746.674290] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 746.763559] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 748.336112] Key type lgssc registered [ 748.693593] Lustre: Echo OBD driver; http://www.lustre.org/ [ 751.262346] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 751.269770] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 752.431494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 753.738423] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 756.187188] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 758.445243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 758.549089] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 760.523802] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 762.661084] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 762.679821] Lustre: 63482:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 762.682952] Lustre: 63482:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 762.685156] Lustre: 63482:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 762.688019] Lustre: 63482:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 762.690873] Lustre: 63482:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 762.692983] Lustre: 63482:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 763.076898] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 766.168659] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 773.089516] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 773.089617] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 773.095757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 774.734617] Lustre: server umount lustre-OST0000 complete [ 776.073124] Lustre: server umount lustre-MDT0000 complete [ 779.730343] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 780.747709] Key type lgssc unregistered [ 780.869426] LNet: 65038:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 780.871568] LNetError: 65038:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 780.879272] LNet: Removed LNI 192.168.206.129@tcp [ 781.186314] Key type .llcrypt unregistered [ 781.187234] Key type ._llcrypt unregistered [ 787.120456] Key type ._llcrypt registered [ 787.122053] Key type .llcrypt registered [ 787.162859] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 793.593524] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 793.908733] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 793.917059] alg: No test for adler32 (adler32-zlib) [ 794.779050] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 794.877947] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 796.456119] Key type lgssc registered [ 796.877371] Lustre: Echo OBD driver; http://www.lustre.org/ [ 808.679889] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 808.685174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 809.770473] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 809.780887] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 809.818172] Lustre: lustre-MDT0000: new disk, initializing [ 809.840221] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 809.849511] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 811.245829] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 815.321927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 815.365316] Lustre: 68161:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 815.390249] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 815.392018] Lustre: Skipped 1 previous similar message [ 815.428083] Lustre: lustre-MDT0001: new disk, initializing [ 815.461805] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 815.475622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 815.478772] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 816.767686] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 820.598986] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 822.447559] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 824.618845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 824.715230] Lustre: lustre-OST0000: new disk, initializing [ 824.717317] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 824.720442] Lustre: 70128:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 824.746133] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 825.046705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 825.049545] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 825.087103] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 826.677385] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 830.227800] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 831.628707] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 831.705796] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 833.018347] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 833.092260] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 835.554570] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 835.554970] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 835.556640] LustreError: Skipped 1 previous similar message [ 835.560514] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 835.563350] Lustre: Skipped 1 previous similar message [ 840.674967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 840.677297] Lustre: Skipped 1 previous similar message [ 845.796644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 845.799756] Lustre: Skipped 1 previous similar message [ 847.840189] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 847.896570] Lustre: server umount lustre-OST0000 complete [ 850.913640] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 850.914446] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 850.918718] Lustre: Skipped 2 previous similar messages [ 855.327576] Lustre: server umount lustre-MDT0000 complete [ 856.617268] LustreError: 68155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316787 with bad export cookie 5069986685975293070 [ 856.620707] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 856.621227] LustreError: 68155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 856.755525] Lustre: server umount lustre-MDT0001 complete [ 859.611941] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 05:19:50 (1785316790) [ 863.223926] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 875.928214] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 879.277246] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 890.926646] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 891.006335] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 891.018339] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 891.063372] Lustre: lustre-MDT0000: new disk, initializing [ 891.085687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 891.093164] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 892.496827] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 896.616505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 896.653972] Lustre: 76151:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 896.673258] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 896.674651] Lustre: Skipped 1 previous similar message [ 896.709665] Lustre: lustre-MDT0001: new disk, initializing [ 896.738265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 896.751833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 896.755769] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 898.099372] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 901.815071] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 903.699369] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 905.928839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 906.025243] Lustre: lustre-OST0000: new disk, initializing [ 906.026802] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 906.030900] Lustre: 78120:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 906.060167] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 907.960774] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 908.407242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 908.412154] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 908.444838] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 911.760622] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 913.162189] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 913.230116] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 914.581702] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 914.649140] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 917.326580] Lustre: server umount lustre-OST0000 complete [ 919.010994] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 919.011947] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 919.015643] Lustre: Skipped 1 previous similar message [ 919.020184] Lustre: Skipped 2 previous similar messages [ 928.254311] Lustre: server umount lustre-MDT0000 complete [ 929.249497] LustreError: 76162:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 929.254462] LustreError: 76162:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 929.580157] LustreError: 76143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316860 with bad export cookie 5069986685975293938 [ 929.582917] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 929.584112] LustreError: 76143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 940.054739] Lustre: server umount lustre-MDT0001 complete [ 942.751174] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:21:13 (1785316873) [ 944.998476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 945.165022] LustreError: 80528:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 945.187195] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 946.622794] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 949.461138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 950.968743] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 953.441951] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 955.250084] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 957.499513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 959.525707] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 961.934121] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 963.158375] Lustre: 83198:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 8cf66b34-4451-4587-a2f5-bc48474b2ee5 at adminstrative request [ 965.092829] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 965.096557] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 975.332389] Lustre: MGS: Client 8cf66b34-4451-4587-a2f5-bc48474b2ee5 (at 0@lo) reconnecting [ 975.335941] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x465c360769847356 to 0x465c3607698475e1 [ 975.343340] Lustre: MGC192.168.206.129@tcp: Connection restored to 0@lo (at 0@lo) [ 995.548729] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 995.625641] Lustre: lustre-OST0001: new disk, initializing [ 995.628485] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 995.632749] Lustre: 83590:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 995.667353] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 995.669512] Lustre: Skipped 2 previous similar messages [ 997.677120] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1001.439445] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1002.449832] Lustre: server umount lustre-OST0001 complete [ 1002.491833] LustreError: 82349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1006.050208] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1006.050249] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1006.053699] LustreError: Skipped 1 previous similar message [ 1006.058474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1006.062600] Lustre: Skipped 3 previous similar messages [ 1007.589948] LustreError: 82349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1007.595235] LustreError: 82349:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1012.708834] LustreError: 82347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1012.715508] LustreError: 82347:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1017.824126] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1017.874693] Lustre: server umount lustre-OST0000 complete [ 1021.408785] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1021.411282] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1021.414937] Lustre: Skipped 1 previous similar message [ 1022.945298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1022.947520] Lustre: Skipped 8 previous similar messages [ 1025.241146] Lustre: server umount lustre-MDT0000 complete [ 1026.482259] LustreError: 80509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316957 with bad export cookie 5069986685975295457 [ 1026.485604] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1026.486327] LustreError: 80509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1026.598924] Lustre: server umount lustre-MDT0001 complete [ 1029.385967] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:22:39 (1785316959) [ 1029.930332] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1030.518729] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:22:41 (1785316961) [ 1031.044958] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1031.640339] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:22:42 (1785316962) [ 1032.211463] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1032.808234] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:22:43 (1785316963) [ 1033.347881] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1033.922762] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:22:44 (1785316964) [ 1034.480112] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1036.492083] loop: module loaded [ 1036.496833] loop0: detected capacity change from 0 to 4294967296 [ 1036.509535] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1037.021195] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1038.146199] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1039.532029] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1048.432220] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1048.446285] LustreError: 87705:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1048.448331] LustreError: 87705:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1048.450438] LustreError: 87705:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1048.452764] LustreError: 87705:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1054.696582] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 1057.982811] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1069.781751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1069.875086] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1069.886535] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1069.923172] Lustre: lustre-MDT0000: new disk, initializing [ 1069.942887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1069.949704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1071.284917] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1075.435522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1075.476142] Lustre: 90562:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1075.568235] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1076.917090] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1080.569640] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1082.429492] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1084.702718] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1084.806062] Lustre: 92531:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1086.009100] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1086.044448] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1086.770260] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1090.576291] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1092.062091] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1092.142517] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1093.627270] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1093.695333] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1096.163036] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1096.163764] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1096.165392] LustreError: Skipped 1 previous similar message [ 1096.168496] Lustre: Skipped 1 previous similar message [ 1096.168973] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1096.172730] Lustre: Skipped 1 previous similar message [ 1100.436659] Lustre: server umount lustre-OST0000 complete [ 1106.401777] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1107.803920] Lustre: server umount lustre-MDT0000 complete [ 1109.039233] LustreError: 90557:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317039 with bad export cookie 5069986685975296066 [ 1109.042524] LustreError: 90557:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1109.042581] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1109.164001] Lustre: server umount lustre-MDT0001 complete [ 1111.802079] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:24:02 (1785317042) [ 1113.997349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1114.146395] LustreError: 94939:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1114.151698] LustreError: 94939:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1115.518584] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1118.422256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1119.869277] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1122.297150] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1124.105258] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1126.413315] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1128.392906] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1130.748292] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1132.222575] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1140.462267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 8 sec [ 1141.876600] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1141.953435] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1144.259438] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1144.336455] Lustre: lustre-OST0001: new disk, initializing [ 1144.338205] Lustre: Skipped 2 previous similar messages [ 1144.340457] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1144.342453] Lustre: Skipped 3 previous similar messages [ 1144.345358] Lustre: 98362:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1146.393168] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1149.947457] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1149.949860] Lustre: Skipped 2 previous similar messages [ 1149.951826] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1149.981624] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1150.102750] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1151.499092] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1151.574544] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1153.034855] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1153.105779] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1163.253206] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1168.416346] Lustre: Failing over lustre-MDT0000 [ 1168.800977] Lustre: server umount lustre-MDT0000 complete [ 1169.888872] LustreError: 94934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1169.890215] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1169.895154] LustreError: 94934:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1169.897483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1169.905888] Lustre: Skipped 3 previous similar messages [ 1183.741074] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1183.742317] LDISKFS-fs (dm-0): recovery complete [ 1183.744918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1183.781901] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1183.914031] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1185.194633] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1185.243512] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1185.243528] Lustre: 94934:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1189.346186] Lustre: 94940:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1189.351252] Lustre: 94940:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 1189.356994] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1189.373616] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1189.688076] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1190.001931] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1190.005463] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1190.320067] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1190.322341] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1190.642784] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1190.646290] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 8 previous similar messages [ 1191.274156] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1191.277381] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1191.592087] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1191.594188] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1192.848883] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1192.853400] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 1193.480923] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1193.484299] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1193.800102] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1193.804124] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1196.930284] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1196.934249] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 1197.562305] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1197.565347] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1197.880084] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1197.882939] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1205.066346] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1205.069332] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 1205.698336] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1205.701467] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1206.016062] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1206.018766] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1221.321925] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1221.325455] Lustre: 101185:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 51 previous similar messages [ 1221.954160] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1221.957594] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 51 previous similar messages [ 1222.272077] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1222.275133] LustreError: 101185:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 51 previous similar messages [ 1245.056118] Lustre: 101185:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1245.059198] Lustre: 101185:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1245.064054] Lustre: 101185:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client da41e100-43ed-403e-b3f1-f2eb16fc16a3@192.168.206.29@tcp [ 1245.067896] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1245.070150] Lustre: 101185:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@000000002a86203a x1872040264009088/t0(8589934775) o101->da41e100-43ed-403e-b3f1-f2eb16fc16a3@192.168.206.29@tcp:396/0 lens 592/0 e 0 to 0 dl 1785317186 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1245.078948] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1245.082118] Lustre: lustre-MDT0000: Denying connection for new client da41e100-43ed-403e-b3f1-f2eb16fc16a3 (at 192.168.206.29@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1245.085034] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1245.100042] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1245.100056] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1245.104192] Lustre: Skipped 2 previous similar messages [ 1245.123038] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 1245.123116] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 1250.790441] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1250.791934] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1250.795059] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1250.795067] Lustre: Skipped 2 previous similar messages [ 1250.796217] Lustre: Skipped 4 previous similar messages [ 1251.903358] Lustre: server umount lustre-OST0000 complete [ 1255.906953] LustreError: 99046:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1255.911608] LustreError: 99046:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 18 previous similar messages [ 1259.285530] Lustre: server umount lustre-OST0001 complete [ 1266.718370] Lustre: server umount lustre-MDT0000 complete [ 1267.944291] LustreError: 94965:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317198 with bad export cookie 5069986685975347250 [ 1267.946199] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1267.950298] LustreError: 94965:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1268.071462] Lustre: server umount lustre-MDT0001 complete [ 1270.931454] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:26:41 (1785317201) [ 1273.378976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1273.381651] Lustre: *** cfs_fail_loc=197, val=0*** [ 1288.800573] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1293.920498] LustreError: 103154:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.129@tcp: failed processing log, type 4: rc = -110 [ 1299.046692] Lustre: *** cfs_fail_loc=197, val=0*** [ 1299.048225] Lustre: Skipped 65 previous similar messages [ 1319.520368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1319.522357] Lustre: Skipped 7 previous similar messages [ 1321.367999] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1323.759754] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1324.483528] Lustre: server umount lustre-OST0000 complete [ 1327.207702] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:27:37 (1785317257) [ 1340.795772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1340.959035] LustreError: 104770:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1342.321675] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1346.101836] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1348.096428] LustreError: 104804:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1348.100555] LustreError: 104804:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1348.105590] LustreError: 104804:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -5 [ 1348.229569] Lustre: server umount lustre-MDT0000 complete [ 1359.034885] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 1362.336154] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1374.325490] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1374.406299] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1374.416434] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1374.453543] Lustre: lustre-MDT0000: new disk, initializing [ 1374.479821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1375.850202] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1379.984056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1380.030862] Lustre: 108966:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1380.144115] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1381.553704] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1385.081126] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1386.813416] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1388.939693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1389.033354] Lustre: 110934:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1390.795170] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1394.271479] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1395.576768] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1398.293274] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1398.694307] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1399.901842] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1399.963081] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1403.362487] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1403.363387] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1403.365107] LustreError: Skipped 2 previous similar messages [ 1403.365530] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1403.370535] Lustre: Skipped 5 previous similar messages [ 1403.371856] Lustre: Skipped 7 previous similar messages [ 1406.669856] Lustre: server umount lustre-OST0000 complete [ 1422.304121] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1422.370417] Lustre: server umount lustre-MDT0000 complete [ 1423.577900] LustreError: 108959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317354 with bad export cookie 5069986685975357876 [ 1423.580385] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1423.582408] LustreError: 108959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1423.691100] Lustre: server umount lustre-MDT0001 complete [ 1426.289340] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:29:16 (1785317356) [ 1429.063727] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 1430.070469] Key type lgssc unregistered [ 1430.198434] LNet: 113441:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1430.200886] LNetError: 113441:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1430.206375] LNet: Removed LNI 192.168.206.129@tcp [ 1430.507415] Key type .llcrypt unregistered [ 1430.508383] Key type ._llcrypt unregistered [ 1435.972179] Key type ._llcrypt registered [ 1435.973018] Key type .llcrypt registered [ 1436.003863] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 1441.909637] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1442.227073] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1442.233109] alg: No test for adler32 (adler32-zlib) [ 1443.093038] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 1443.189926] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 1444.760125] Key type lgssc registered [ 1445.120179] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1458.374947] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1458.385447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1459.475611] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1459.486106] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1459.522927] Lustre: lustre-MDT0000: new disk, initializing [ 1459.546908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1459.555337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1460.936414] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1465.170173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1465.277227] Lustre: lustre-OST0000: new disk, initializing [ 1465.279349] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1465.281264] Lustre: Skipped 1 previous similar message [ 1465.283084] Lustre: 117602:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1465.306510] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1466.947466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1466.953474] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1466.962199] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1467.288497] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1471.442637] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1471.500344] Lustre: lustre-OST0001: new disk, initializing [ 1471.501935] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1471.504496] Lustre: 118466:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1471.529298] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1472.458928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1472.461630] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1472.471096] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1473.547545] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1477.603109] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1477.603532] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1477.606177] Lustre: Skipped 1 previous similar message [ 1482.721637] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1482.724017] Lustre: Skipped 2 previous similar messages [ 1483.758601] Lustre: server umount lustre-MDT0000 complete [ 1485.970800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1487.532251] LustreError: 117601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317418 with bad export cookie 13159400319576286204 [ 1487.534817] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1487.537344] LustreError: 117601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1487.569045] Lustre: server umount lustre-OST0000 complete [ 1488.838570] Lustre: server umount lustre-OST0001 complete [ 1491.729693] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:30:22 (1785317422) [ 1498.874683] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:30:29 (1785317429) [ 1501.712085] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1508.490765] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:30:39 (1785317439) [ 1520.710104] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 1524.113055] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1535.919322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1536.003477] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1536.014850] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1536.054303] Lustre: lustre-MDT0000: new disk, initializing [ 1536.075472] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1536.080815] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1537.410242] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1541.481915] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1541.525267] Lustre: 126200:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1541.548655] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1541.550448] Lustre: Skipped 1 previous similar message [ 1541.584501] Lustre: lustre-MDT0001: new disk, initializing [ 1541.616081] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1541.626779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1541.630672] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1542.956914] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1546.705230] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1548.667396] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1550.988069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1551.099840] Lustre: lustre-OST0000: new disk, initializing [ 1551.101362] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1551.104311] Lustre: 128168:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1551.140566] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1553.186618] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1554.424366] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1554.428477] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1554.458197] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1557.037929] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1558.599595] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1558.690894] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1560.137723] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1560.214975] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1562.898304] Lustre: server umount lustre-OST0000 complete [ 1564.642358] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1564.643118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1564.648063] Lustre: Skipped 1 previous similar message [ 1569.761288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1569.764308] Lustre: Skipped 2 previous similar messages [ 1574.881922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1574.884626] Lustre: Skipped 1 previous similar message [ 1574.928313] Lustre: server umount lustre-MDT0000 complete [ 1576.216314] LustreError: 126193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317507 with bad export cookie 13159400319576287415 [ 1576.221185] LustreError: 126193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1576.221579] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1586.706138] Lustre: server umount lustre-MDT0001 complete [ 1589.666186] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:32:00 (1785317520) [ 1591.954513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1592.151154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1593.604207] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1596.513959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1597.985799] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1600.430143] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1602.348130] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1604.725995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1606.850466] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1609.358103] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1621.971296] Lustre: *** cfs_fail_loc=159, val=0*** [ 1638.374596] Lustre: lustre-MDT0000: Client a9c33354-1b74-49f0-8a8e-cfae8d8e4690 (at 192.168.206.29@tcp) reconnecting [ 1638.913120] Lustre: *** cfs_fail_loc=159, val=0*** [ 1638.914661] Lustre: Skipped 5 previous similar messages [ 1654.248561] Lustre: lustre-MDT0000: Client a9c33354-1b74-49f0-8a8e-cfae8d8e4690 (at 192.168.206.29@tcp) reconnecting [ 1658.342579] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1658.343181] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1658.344916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1658.346042] Lustre: Skipped 1 previous similar message [ 1658.352403] LustreError: Skipped 1 previous similar message [ 1661.584871] Lustre: server umount lustre-OST0000 complete [ 1663.457750] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1663.458237] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1677.280114] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1677.337219] Lustre: server umount lustre-MDT0000 complete [ 1678.539416] LustreError: 130559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317609 with bad export cookie 13159400319576288283 [ 1678.543623] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1678.544511] LustreError: 130559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1678.657521] Lustre: server umount lustre-MDT0001 complete [ 1681.867447] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 1682.784836] Key type lgssc unregistered [ 1682.894344] LNet: 134373:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1682.896453] LNetError: 134373:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1682.904221] LNet: Removed LNI 192.168.206.129@tcp [ 1683.183770] Key type .llcrypt unregistered [ 1683.184685] Key type ._llcrypt unregistered [ 1691.898764] Key type ._llcrypt registered [ 1691.900147] Key type .llcrypt registered [ 1691.934411] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1692.338722] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1692.354039] alg: No test for adler32 (adler32-zlib) [ 1693.195104] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 1693.276358] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 1694.848244] Key type lgssc registered [ 1695.153130] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1697.692569] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1697.698600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1698.852416] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1700.245274] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1703.168630] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1703.292872] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1704.735178] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1707.277208] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1709.190818] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1711.558713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1711.690658] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1713.663246] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1716.214262] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1724.006987] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1731.539345] Lustre: *** cfs_fail_loc=159, val=0*** [ 1747.942495] Lustre: lustre-MDT0000: Client 9dfbddbe-01ca-4a2e-ad93-9b4bf24506b7 (at 192.168.206.29@tcp) reconnecting [ 1748.658097] Lustre: *** cfs_fail_loc=159, val=0*** [ 1764.842578] Lustre: lustre-MDT0001: Client 9dfbddbe-01ca-4a2e-ad93-9b4bf24506b7 (at 192.168.206.29@tcp) reconnecting [ 1765.355784] Lustre: *** cfs_fail_loc=159, val=0*** [ 1765.358430] Lustre: Skipped 4 previous similar messages [ 1781.734692] Lustre: lustre-MDT0001: Client 9dfbddbe-01ca-4a2e-ad93-9b4bf24506b7 (at 192.168.206.29@tcp) reconnecting [ 1796.226949] Lustre: 138840:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1797.048150] Lustre: *** cfs_fail_loc=159, val=0*** [ 1797.049920] Lustre: Skipped 3 previous similar messages [ 1813.481438] Lustre: lustre-MDT0000: Client 7edaccd7-38b0-44b6-8ed3-5563af0f97f4 (at 192.168.206.29@tcp) reconnecting [ 1814.072112] Lustre: *** cfs_fail_loc=159, val=0*** [ 1814.073523] Lustre: Skipped 14 previous similar messages [ 1830.378923] Lustre: lustre-MDT0000: Client 7edaccd7-38b0-44b6-8ed3-5563af0f97f4 (at 192.168.206.29@tcp) reconnecting [ 1830.994465] Lustre: 139131:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1849.827894] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1849.830996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1849.832029] Lustre: Skipped 1 previous similar message [ 1854.947462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1854.950768] Lustre: Skipped 2 previous similar messages [ 1860.068448] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1860.070694] Lustre: Skipped 1 previous similar message [ 1862.112153] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1862.166156] Lustre: server umount lustre-OST0000 complete [ 1865.185665] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1865.190772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1865.192784] Lustre: Skipped 1 previous similar message [ 1869.658757] Lustre: server umount lustre-MDT0000 complete [ 1870.305115] LustreError: 136489:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1870.312836] LustreError: 136489:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 1870.991427] LustreError: 135745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317801 with bad export cookie 12445101169371748010 [ 1870.994844] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1870.997220] LustreError: 135745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1871.119670] Lustre: server umount lustre-MDT0001 complete [ 1875.081753] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 1876.295467] Key type lgssc unregistered [ 1876.434416] LNet: 140208:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1876.438918] LNetError: 140208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1876.447336] LNet: Removed LNI 192.168.206.129@tcp [ 1876.794112] Key type .llcrypt unregistered [ 1876.795514] Key type ._llcrypt unregistered [ 1886.843124] Key type ._llcrypt registered [ 1886.844208] Key type .llcrypt registered [ 1886.892586] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1887.276156] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1887.285437] alg: No test for adler32 (adler32-zlib) [ 1888.143681] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 1888.252473] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 1889.848203] Key type lgssc registered [ 1890.341622] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1893.066131] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1893.073336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1894.226771] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1895.663530] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1898.673192] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1898.802535] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1900.226478] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1902.643205] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1904.415103] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1906.668348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1906.776704] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1908.674650] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1908.843730] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1911.201776] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1920.106101] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 1924.238740] Lustre: 144202:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1924.931213] Lustre: 144250:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1925.434484] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 1926.584575] Lustre: 144397:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 1926.893689] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 1926.896049] Lustre: Skipped 1 previous similar message [ 1927.192984] Lustre: 144495:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1929.697480] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1929.699782] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1929.703526] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1930.723611] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1930.723904] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1930.723918] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1934.282839] Lustre: server umount lustre-OST0000 complete [ 1935.841239] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1935.841788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1935.844487] Lustre: Skipped 1 previous similar message [ 1935.846829] Lustre: Skipped 1 previous similar message [ 1940.961298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1940.963415] Lustre: Skipped 1 previous similar message [ 1941.590681] Lustre: server umount lustre-MDT0000 complete [ 1942.748122] LustreError: 143442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317873 with bad export cookie 13070606472151575703 [ 1942.750706] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1942.751284] LustreError: 143442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1942.869455] Lustre: server umount lustre-MDT0001 complete [ 1946.250863] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 1947.157629] Key type lgssc unregistered [ 1947.286301] LNet: 145554:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1947.288301] LNetError: 145554:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1947.299220] LNet: Removed LNI 192.168.206.129@tcp [ 1947.573156] Key type .llcrypt unregistered [ 1947.574095] Key type ._llcrypt unregistered [ 1956.165854] Key type ._llcrypt registered [ 1956.166852] Key type .llcrypt registered [ 1956.206174] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 1956.606576] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1956.616458] alg: No test for adler32 (adler32-zlib) [ 1957.465490] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 1957.561799] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 1959.144155] Key type lgssc registered [ 1959.486968] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1962.156737] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1962.162936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1963.297978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1964.612849] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1967.374141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1967.486753] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1968.732764] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1971.040731] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1972.790551] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1974.838030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1974.968159] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1976.827403] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 1978.026265] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 1979.279453] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1988.263327] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 1991.770778] Lustre: *** cfs_fail_loc=159, val=0*** [ 2007.019368] Lustre: lustre-MDT0000: Client 2c626277-be90-4840-903c-e7a9fa025caf (at 192.168.206.29@tcp) reconnecting [ 2024.420604] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2024.421358] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2024.428045] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2024.430895] Lustre: Skipped 1 previous similar message [ 2028.952500] Lustre: server umount lustre-OST0000 complete [ 2034.144989] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2034.147563] LustreError: Skipped 1 previous similar message [ 2034.148931] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2034.152873] Lustre: Skipped 1 previous similar message [ 2034.155058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2036.377755] Lustre: server umount lustre-MDT0000 complete [ 2037.745157] LustreError: 146974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317968 with bad export cookie 2013495252797044236 [ 2037.748257] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2037.749659] LustreError: 146974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2037.880398] Lustre: server umount lustre-MDT0001 complete [ 2041.462031] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 2042.522956] Key type lgssc unregistered [ 2042.654282] LNet: 150629:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2042.656582] LNetError: 150629:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2042.664252] LNet: Removed LNI 192.168.206.129@tcp [ 2042.955262] Key type .llcrypt unregistered [ 2042.956307] Key type ._llcrypt unregistered [ 2051.843501] Key type ._llcrypt registered [ 2051.844354] Key type .llcrypt registered [ 2051.876134] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 2052.175450] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2052.228370] alg: No test for adler32 (adler32-zlib) [ 2053.092148] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 2053.196248] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 2054.776153] Key type lgssc registered [ 2055.181761] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2057.818490] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2057.826452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2058.971560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2060.381946] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2063.171445] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2063.286204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2064.576560] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2067.130702] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2069.152412] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2071.534746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2071.700445] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2073.767783] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2073.822801] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2076.425930] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2084.005577] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2099.735609] Lustre: lustre-OST0000: evicting 09043774-3b97-4393-b1cc-c89ed036471a (at 192.168.206.29@tcp) by administrative request [ 2099.738552] Lustre: Skipped 1 previous similar message [ 2125.281091] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2125.281231] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2125.284943] LustreError: Skipped 1 previous similar message [ 2125.288908] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2125.291422] Lustre: Skipped 1 previous similar message [ 2128.353411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2128.355113] Lustre: Skipped 1 previous similar message [ 2133.473788] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2133.475605] Lustre: Skipped 1 previous similar message [ 2138.080216] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2138.138932] Lustre: server umount lustre-OST0000 complete [ 2140.128796] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2140.130993] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2140.134312] Lustre: Skipped 1 previous similar message [ 2140.135983] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2143.713908] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2145.616700] Lustre: server umount lustre-MDT0000 complete [ 2146.855302] LustreError: 151999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318077 with bad export cookie 17592072477892780920 [ 2146.859572] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2146.981475] Lustre: server umount lustre-MDT0001 complete [ 2150.850852] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 2151.952328] Key type lgssc unregistered [ 2152.088534] LNet: 156107:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2152.092283] LNetError: 156107:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2152.102387] LNet: Removed LNI 192.168.206.129@tcp [ 2152.443723] Key type .llcrypt unregistered [ 2152.445310] Key type ._llcrypt unregistered [ 2176.182754] Key type ._llcrypt registered [ 2176.183627] Key type .llcrypt registered [ 2176.220612] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 2176.564787] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2176.592285] alg: No test for adler32 (adler32-zlib) [ 2177.440061] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 2177.533353] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 2179.112167] Key type lgssc registered [ 2179.536471] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2182.346375] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2182.353894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2183.511408] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2184.886552] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2187.824545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2187.987218] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2189.543095] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2192.085575] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2194.033771] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2196.375575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2196.498873] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2197.544706] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2198.434646] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2200.935922] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2208.806293] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2217.953235] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2217.955366] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2217.959737] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2218.980287] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2218.982211] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2218.982309] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2218.987945] Lustre: Skipped 1 previous similar message [ 2224.101513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2224.103276] Lustre: Skipped 1 previous similar message [ 2228.192122] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2228.241942] Lustre: server umount lustre-OST0000 complete [ 2234.338715] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2234.339615] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2234.339847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2234.342438] Lustre: Skipped 1 previous similar message [ 2235.679018] Lustre: server umount lustre-MDT0000 complete [ 2236.986326] LustreError: 161072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318167 with bad export cookie 17121086848578751720 [ 2236.988823] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2236.991949] LustreError: 161072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2237.115776] Lustre: server umount lustre-MDT0001 complete [ 2241.085393] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 2242.206791] Key type lgssc unregistered [ 2242.359525] LNet: 162126:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2242.363615] LNetError: 162126:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2242.376423] LNet: Removed LNI 192.168.206.129@tcp [ 2242.743317] Key type .llcrypt unregistered [ 2242.745234] Key type ._llcrypt unregistered [ 2252.414596] Key type ._llcrypt registered [ 2252.416066] Key type .llcrypt registered [ 2252.461722] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 2252.961433] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2252.983140] alg: No test for adler32 (adler32-zlib) [ 2253.849376] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 2253.949917] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 2255.544211] Key type lgssc registered [ 2255.941394] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2261.175540] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:43:11 (1785318191) [ 2266.594781] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 2270.130809] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 2281.972366] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2281.978909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2283.068508] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2283.078892] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2283.124090] Lustre: lustre-MDT0000: new disk, initializing [ 2283.146072] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2283.152894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2284.508516] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2288.714843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2288.763532] Lustre: 166360:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2288.789511] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2288.792475] Lustre: Skipped 1 previous similar message [ 2288.830118] Lustre: lustre-MDT0001: new disk, initializing [ 2288.886234] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2288.901596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2288.907286] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2290.266281] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2294.113084] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2296.107701] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2298.249030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2298.332181] Lustre: lustre-OST0000: new disk, initializing [ 2298.333471] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2298.335342] Lustre: 168328:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2298.359813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2300.153501] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2303.483104] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2303.486479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2303.517775] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2303.859210] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2308.578493] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2308.579582] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2308.580715] LustreError: Skipped 1 previous similar message [ 2308.580754] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2308.584075] Lustre: Skipped 1 previous similar message [ 2309.207938] Lustre: server umount lustre-OST0000 complete [ 2312.541967] Lustre: server umount lustre-MDT0000 complete [ 2313.698233] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2313.703496] Lustre: Skipped 1 previous similar message [ 2313.759437] LustreError: 166352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318244 with bad export cookie 2883511785385904560 [ 2313.764793] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2313.881295] Lustre: server umount lustre-MDT0001 complete [ 2317.397698] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 2318.472213] Key type lgssc unregistered [ 2318.606478] LNet: 170229:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2318.608930] LNetError: 170229:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2318.615340] LNet: Removed LNI 192.168.206.129@tcp [ 2318.957499] Key type .llcrypt unregistered [ 2318.958512] Key type ._llcrypt unregistered [ 2328.439252] Key type ._llcrypt registered [ 2328.441053] Key type .llcrypt registered [ 2328.487238] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 2329.010530] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2329.035175] alg: No test for adler32 (adler32-zlib) [ 2329.889399] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 2329.988844] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 2331.576179] Key type lgssc registered [ 2331.961242] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2334.604187] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2334.609591] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2335.753527] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2337.097057] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2340.030448] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2340.154446] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2341.461935] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2343.871122] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2345.725598] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2347.989523] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2348.111427] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2350.052433] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2352.599935] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2360.423133] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2365.745214] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2365.753216] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2365.757544] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2365.760584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.764489] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2366.154669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.511200] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2366.514096] Lustre: Skipped 1 previous similar message [ 2366.520156] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2366.523770] Lustre: Skipped 1 previous similar message [ 2366.525402] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2366.528170] Lustre: Skipped 1 previous similar message [ 2366.529585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.532921] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2366.534722] Lustre: Skipped 1 previous similar message [ 2366.852372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.149488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.456283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.787985] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2367.791746] Lustre: Skipped 3 previous similar messages [ 2367.799420] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2367.803042] Lustre: Skipped 3 previous similar messages [ 2367.804533] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2367.806723] Lustre: Skipped 3 previous similar messages [ 2367.808645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.811959] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2367.814495] Lustre: Skipped 3 previous similar messages [ 2368.137173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.460509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.780802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.146809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.464847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.765843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.084070] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2370.086635] Lustre: Skipped 6 previous similar messages [ 2370.093098] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2370.097507] Lustre: Skipped 6 previous similar messages [ 2370.099851] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2370.103402] Lustre: Skipped 6 previous similar messages [ 2370.105734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.109583] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2370.112566] Lustre: Skipped 6 previous similar messages [ 2370.433029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.759582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.089246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.397730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.721143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.039498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.361628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.671837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.037617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.378803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.701708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.029818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.364038] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2374.366158] Lustre: Skipped 12 previous similar messages [ 2374.372079] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2374.376138] Lustre: Skipped 12 previous similar messages [ 2374.377620] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2374.380071] Lustre: Skipped 12 previous similar messages [ 2374.381837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.385819] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2374.387709] Lustre: Skipped 12 previous similar messages [ 2374.691129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.995700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.293157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.622633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.937043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.246259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.559903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.863934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.168886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.461901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.763895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.064677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.367433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.671184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.968372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.280562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.592476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.904237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.226444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.548949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.845805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.145670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.466213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.827838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.162763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.474279] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2382.476586] Lustre: Skipped 25 previous similar messages [ 2382.481823] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2382.485588] Lustre: Skipped 25 previous similar messages [ 2382.487185] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2382.489954] Lustre: Skipped 25 previous similar messages [ 2382.491574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.496365] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2382.498167] Lustre: Skipped 25 previous similar messages [ 2382.807358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.144750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.458127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.773606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.084594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.432128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.773783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.106397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.434030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.762897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.074681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.378921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.665129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.984593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.275377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.569221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.861609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.166204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.458590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.778847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.072336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.358567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.662964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.969516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.278764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.582911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.888103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.214813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.537989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.848756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.156992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.472945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.775256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.084209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.401484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.715932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.029944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.363853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.674933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.998965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.300315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.600340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.907333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.226126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.538695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.856402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.174637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.508198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.828866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.131983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.431809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.733034] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2398.736540] Lustre: Skipped 51 previous similar messages [ 2398.744013] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2398.747277] Lustre: Skipped 51 previous similar messages [ 2398.748863] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2398.750712] Lustre: Skipped 51 previous similar messages [ 2398.752135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.756620] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2398.759869] Lustre: Skipped 51 previous similar messages [ 2399.042101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.347463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.645530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.959637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.260647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.572111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.877552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.204467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.517353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.815864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.117135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.417436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.727540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.035303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.360144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.703316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.014811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.330334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.640744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.960673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.279975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.596389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.895887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.200066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.501712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.809203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.113663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.418570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.723633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.019245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.321457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.654944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.972661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.277324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.573414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.910866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.224043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.517616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.820965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.144254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.449712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.752493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.073328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.398055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.709247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.029182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.338912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.663794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.978674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.281038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.591937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.926366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.232489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.541535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.872859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.210812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.518415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.829838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.156451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.495235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.815583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.129119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.423280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.747660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.063632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.406837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.715361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.033083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.342849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.665881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.978087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.300652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.618240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.915473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.248523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.591836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.890236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.205621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.502996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.816436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.110773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.425977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.741208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.046219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.331755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.636240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.942431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.277081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.591616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.902815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.220968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.557886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.867938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.184897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.497736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.842402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.141832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.458812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.796755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.130731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.455425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.757506] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2430.760221] Lustre: Skipped 101 previous similar messages [ 2430.766168] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2430.770591] Lustre: Skipped 101 previous similar messages [ 2430.772387] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2430.774943] Lustre: Skipped 101 previous similar messages [ 2430.776910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.781418] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2430.784130] Lustre: Skipped 101 previous similar messages [ 2431.088811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.412774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.703905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.033360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.346499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.673224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.976743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.291292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.581486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.926106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.210262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.509847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.815448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.132162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.454156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.768545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.092646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.401838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.689510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.973769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.283160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.619749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.938493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.292710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.624202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.945629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.286447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.657843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.997942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.333591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.659615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.977430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.309731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.604470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.890567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.188131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.490346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.774784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.054795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.335057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.644319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.957332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.244875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.553963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.849960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.115422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.382980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.676440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.979137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.280070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.599064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.908695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.236089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.549333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.853687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.160283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.464309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.763027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.070365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.391147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.714536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.024392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.325648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.653417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.993304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.299589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.629394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.955184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.292843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.602413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.914803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.217064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.561191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.888337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.192516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.500766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.845156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.172452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.479612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.758356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.045964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.317302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.603131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.901116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.192798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.493328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.800781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.089500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.372412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.654654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.940937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.221187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.494939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.769726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.038748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.319199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.602518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.895657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.176624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.454163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.740974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.015326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.286023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.554322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.834225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.106866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.387258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.673977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.963779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.269113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.575252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.866481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.159947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.441125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.723301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.014927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.316876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.607835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.911714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.209081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.516544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.794279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.079461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.370611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.649439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.929202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.206061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.493190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.770881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.052420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.332399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.609685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.892450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.178366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.474708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.753870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.052812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.333297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.626920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.903438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.198994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.469207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.753560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.043198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.353602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.652342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.940015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.221886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.507400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.773180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.044651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.306984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.585131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.888244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.181774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.476113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.787363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.091459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.389584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.700459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.005500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.308137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.630304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.921291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.217829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.500716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.780617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.081263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.396020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.693038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.980691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.281692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.609910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.931643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.233249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.524820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.803337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.085201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.367663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.637677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.914987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.200818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.469302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.765474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.047500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.321777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.604563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.883893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.161868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.427524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.698269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.968523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.261887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.533619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.805258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.072475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.376856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.681662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.998638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.320162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.658463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.978104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.299258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.623505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.006026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.332067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.637182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.936771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.263431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.574071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.896289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.206168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.538748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.872306] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2494.874742] Lustre: Skipped 213 previous similar messages [ 2494.881849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2494.888283] Lustre: Skipped 213 previous similar messages [ 2494.890311] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2494.892747] Lustre: Skipped 213 previous similar messages [ 2494.894421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.901198] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2494.904342] Lustre: Skipped 213 previous similar messages [ 2495.245432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.565550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.904514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.213945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.530806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.841028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.161029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.463894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.776379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.097378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.420502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.728151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.036522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.337451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.652245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.977658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.300414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.597619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.910518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.210706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.526609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.840067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.179411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.485724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.805453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.114146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.428942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.738690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.056168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.381788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.725320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.037980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.373401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.690563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.028699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.362937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.670991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.992092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.340849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.651683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.943230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.226115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.548767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.814842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.098934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.393206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.688375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.978173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.245122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.513431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.789754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.090900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.412585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.736111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.040661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.318639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.590041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.859914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.150152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.438865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.722586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.024284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.341069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.626219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.933721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.253744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.561227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.831402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.105931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.410312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.718508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.018475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.321016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.623643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.952876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.252290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.550496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.870068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.189660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.501989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.803703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.108111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.420373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.738668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.058745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.374078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.708994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.019279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.329372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.649533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.961174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.259573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.545782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.855092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.173690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.467255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.763299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.064364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.368182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.686645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.983162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.288398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.592282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.881984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.182848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.471881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.767152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.065393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.359235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.675593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.000379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.303893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.600138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.896098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.232526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.533063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.834489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.145386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.476420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.806352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.114747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.434704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.762775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.074927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.387744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.686448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.022340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.338235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.648580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.990043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.318376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.599361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.886975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.171909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.481902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.782518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.080119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.378737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.698734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.000232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.298651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.563282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.840049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.120579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.396827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.665983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.956433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.229419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.502246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.826369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.146933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.454614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.778047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.090705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.433280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.759163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.078564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.382378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.711181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.026113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.335867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.638683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.958746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.251592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.552516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.841721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.165269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.462639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.766609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.075489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.388417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.684269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.981096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.290287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.606698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.910080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.202890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.500548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.817072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.123416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.461951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.782426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.106641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.429157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.740793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.068963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.409264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.722704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.045436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.364248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.698901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.022735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.342212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.660284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.011950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.342818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.615790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.894201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.189695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.497118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.774752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.064363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.362385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.640358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.912414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.215934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.506269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.787959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.068196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.371424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.691516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.978059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.259509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.524911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.817576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.104103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.386333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.672904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.014299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.320446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.638055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.955885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.284246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.578977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.884938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.185511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.503975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.798224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.082649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.392467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.700367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.993378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.290124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.583160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.893644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.191023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.474410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.767981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.060703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.349253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.618337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.896614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.166746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.431568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.702376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.971463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.257181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.520855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.786714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.071333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.356809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.623503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.907402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.184850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.475715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.759402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.041185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.321115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.617379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.905234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.186869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.461188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.740323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.005989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.277767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.558419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.835189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.121941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.410666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.706504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.019778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.317996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.591667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.880995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.171655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.462849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.785607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.103162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.424388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.743411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.019279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.314961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.606547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.884931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.174431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.466162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.760889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.047956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.329402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.618978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.931624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.222905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.497693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.788491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.087571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.367886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.651980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.933360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.215165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.471936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.729917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.024769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.329349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.632171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.942201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.251053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.561156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.861504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.155133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.453515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.757196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.047158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.336646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.661848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.001300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.313283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.609958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.910908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.236311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.551260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.857280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.166436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.483525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.784225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.082527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.384602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.722585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.050881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.380456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.714936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.053720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.378583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.715650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.035613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.349765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.645887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.946321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.250560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.579187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.898823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.223083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.540250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.881132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.186638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.496725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.804614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.125514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.435039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.754282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.076699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.394091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.698739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.008976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.318496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.647553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.951223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.254664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.554821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.883737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.213471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.528040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.836395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.150845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.462453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.754227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.050172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.365225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.667663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.978790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.298888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.627719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.952097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.261130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.569646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.905016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.223322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.530709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.844327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.178191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.514972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.824987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.134048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.468804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.793996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.084747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.378375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.687245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.961289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.245216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.531310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.812462] LustreError: 171638:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2613.816421] LustreError: 171638:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2613.816884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.113057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.395817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.697228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.997794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.301842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.588919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.871824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.165117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.451491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.761628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.046083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.360192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.657231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.947747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.260933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.575324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.872769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.181139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.490384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.794152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.081301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.378645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.692256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.017842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.328990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.630405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.932148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.278685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.610441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.917680] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2622.919383] Lustre: Skipped 420 previous similar messages [ 2622.925248] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2622.929154] Lustre: Skipped 420 previous similar messages [ 2622.930783] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2622.933192] Lustre: Skipped 420 previous similar messages [ 2622.935073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.938374] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2622.941068] Lustre: Skipped 420 previous similar messages [ 2623.257808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.577348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.880667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.183725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.488522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.815310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.109667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.408415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.749340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.083409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.391872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.692210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.987261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.294933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.600175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.900436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.230378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.554149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.880466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.195574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.483682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.809299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.127777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.446626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.765363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.102738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.394678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.699336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.011906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.328235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.640593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.943898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.262322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.588395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.888539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.183254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.483915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.775733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.068418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.351718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.635379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.933679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.224471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.515382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.801387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.087517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.351622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.625071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.908311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.181251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.470319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.749705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.035933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.330984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.599848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.883951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.156301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.448088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.733212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.014996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.280598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.573305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.864663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.157036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.453367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.762285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.062880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.366435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.672275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.979667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.267464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.554971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.849664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.149121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.423800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.703703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.985260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.284040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.576456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.870918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.162265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.473241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.771763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.066197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.367840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.695957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.012163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.328173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.643686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.967652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.264684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.563164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.856931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.150279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.447260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.741081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.034326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.362482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.672089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.949894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.226844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.510219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.788584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.061345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.339760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.630548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.924426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.218517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.533140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.849279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.150920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.450673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.739496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.042935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.332483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.613189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.904568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.229875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.549898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.844877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.147348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.503215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.834737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.167291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.524975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.862253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.223370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.569556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.924541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.246129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.585031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.914223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.259636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.598425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.953603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.317389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.667693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.025563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.377691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.739451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.070524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.418390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.746963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.074127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.416165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.758399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.072117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.395144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.745861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.076679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.408875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.723042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.019467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.333088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.662561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.984851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.287163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.601298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.546345] LustreError: 173440:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2738.549216] LustreError: 171638:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 2805.221417] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2805.224092] Lustre: Skipped 1 previous similar message [ 2805.518607] Lustre: server umount lustre-OST0000 complete [ 2810.339554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2810.342687] Lustre: Skipped 1 previous similar message [ 2813.263526] Lustre: server umount lustre-MDT0000 complete [ 2814.561153] LustreError: 171601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318745 with bad export cookie 16018414432180910863 [ 2814.564126] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2814.564910] LustreError: 171601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2814.687829] Lustre: server umount lustre-MDT0001 complete [ 2818.558982] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 2819.554312] Key type lgssc unregistered [ 2819.677386] LNet: 226637:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2819.680454] LNetError: 226637:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2819.692326] LNet: Removed LNI 192.168.206.129@tcp [ 2819.988339] Key type .llcrypt unregistered [ 2819.989462] Key type ._llcrypt unregistered [ 2829.762035] Key type ._llcrypt registered [ 2829.763133] Key type .llcrypt registered [ 2829.808604] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 2830.299835] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2830.315799] alg: No test for adler32 (adler32-zlib) [ 2831.180550] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 2831.281987] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 2832.872179] Key type lgssc registered [ 2833.248136] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2835.989243] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2835.995994] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2837.155056] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2838.618619] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2841.473896] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2841.601282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2843.006490] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2845.403139] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2847.246580] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2849.564629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2849.689876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2851.693027] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2854.276720] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2863.021281] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 2866.549845] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2867.187583] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2867.193954] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2867.197655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.202195] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2867.202218] LustreError: 228049:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 2868.194743] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2868.200300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2868.202498] Lustre: Skipped 1 previous similar message [ 2869.729259] Lustre: lustre-OST0000: Not available for connect from 192.168.206.29@tcp (stopping) [ 2869.917293] Lustre: server umount lustre-OST0000 complete [ 2887.280148] LustreError: 228049:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 2897.836957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2897.939843] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2899.730982] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2901.993904] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2908.194194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.197080] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.197091] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2908.202451] Lustre: Skipped 1 previous similar message [ 2910.614212] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 8 sec [ 2912.444234] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9377b8ac9800.ost_server_uuid 50 [ 2912.951389] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9377b8ac9800.ost_server_uuid in FULL state after 0 sec [ 2918.371797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2918.371860] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2918.374854] Lustre: Skipped 1 previous similar message [ 2918.378625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2918.381996] Lustre: Skipped 1 previous similar message [ 2920.010761] Lustre: server umount lustre-OST0000 complete [ 2923.489687] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2923.490162] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2923.490624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2923.500792] Lustre: Skipped 1 previous similar message [ 2927.313246] Lustre: server umount lustre-MDT0000 complete [ 2928.471859] LustreError: 228011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318859 with bad export cookie 10761259164064059578 [ 2928.474441] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2928.476095] LustreError: 228011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2928.576716] Lustre: server umount lustre-MDT0001 complete [ 2932.252262] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 2933.221423] Key type lgssc unregistered [ 2933.342323] LNet: 233252:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2933.345199] LNetError: 233252:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2933.359274] LNet: Removed LNI 192.168.206.129@tcp [ 2933.659809] Key type .llcrypt unregistered [ 2933.660868] Key type ._llcrypt unregistered [ 2944.531849] Key type ._llcrypt registered [ 2944.532938] Key type .llcrypt registered [ 2944.568458] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 2944.936770] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2944.993389] alg: No test for adler32 (adler32-zlib) [ 2945.854322] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 2945.944365] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 2947.520085] Key type lgssc registered [ 2947.846578] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2950.728300] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2950.733390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2951.867718] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2953.101294] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2955.869075] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2955.993346] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2957.300518] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2958.221413] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 2960.266291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2960.376405] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2961.384324] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 2962.234298] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2964.846621] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2964.912128] Lustre: lustre-OST0001: new disk, initializing [ 2964.914289] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2964.917576] Lustre: 237264:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2964.950330] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2966.712069] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 2971.646153] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2971.651093] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2971.675228] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2971.952115] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2975.575329] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 2981.857266] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2981.859230] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2981.862259] Lustre: Skipped 3 previous similar messages [ 2981.862329] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2981.865607] Lustre: Skipped 3 previous similar messages [ 2986.978892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2986.980904] Lustre: Skipped 3 previous similar messages [ 2987.993892] Lustre: server umount lustre-MDT0000 complete [ 2989.121263] LustreError: 234937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318920 with bad export cookie 11643238605109126039 [ 2989.124339] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2989.126073] LustreError: 234937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2989.231817] Lustre: server umount lustre-MDT0001 complete [ 3000.321702] Lustre: server umount lustre-OST0000 complete [ 3011.674230] Lustre: server umount lustre-OST0001 complete [ 3018.353433] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3021.727140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3021.891053] LustreError: 240360:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3021.898597] LustreError: 240360:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3021.912664] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3023.408343] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3026.399044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3027.986807] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3028.993990] Lustre: 241502:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3031.233638] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3031.366643] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3031.369389] Lustre: Skipped 1 previous similar message [ 3032.421180] LustreError: 241855:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3032.425558] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 3033.396890] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3036.279467] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3037.419758] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 3038.145706] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3041.207706] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3042.353973] Lustre: 243367:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3046.880886] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3046.884108] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3046.889519] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3047.907655] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3047.912768] Lustre: Skipped 2 previous similar messages [ 3051.742136] Lustre: server umount mylustre-MDT0000 complete [ 3052.935792] LustreError: 243370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318983 with bad export cookie 11643238605109128748 [ 3052.941447] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3053.025384] LustreError: 240356:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3053.026338] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3053.030899] LustreError: 240356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3053.072489] Lustre: server umount mylustre-MDT0001 complete [ 3060.475257] Lustre: server umount mylustre-OST0000 complete [ 3061.671859] Lustre: server umount mylustre-OST0001 complete [ 3067.932482] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3071.279965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3071.411848] LustreError: 245672:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3071.428931] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3071.431322] Lustre: Skipped 1 previous similar message [ 3072.813076] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3075.462949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3076.841798] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3077.781507] Lustre: 246815:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3079.930022] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3081.778729] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3082.084920] LustreError: 247170:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3082.087713] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 3084.419748] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3085.548956] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 3085.550253] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 3086.246994] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3089.183172] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3090.292936] Lustre: 248682:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3096.035535] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3096.036899] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3096.037509] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3096.037527] Lustre: Skipped 4 previous similar messages [ 3096.039586] Lustre: Skipped 4 previous similar messages [ 3098.332396] Lustre: server umount tfs-MDT0000 complete [ 3099.500220] LustreError: 245648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319030 with bad export cookie 11643238605109130701 [ 3099.503829] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3099.503981] LustreError: 245648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 3099.600670] Lustre: server umount tfs-MDT0001 complete [ 3110.346706] Lustre: server umount tfs-OST0000 complete [ 3121.757627] Lustre: server umount tfs-OST0001 complete [ 3128.479265] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3131.910052] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3132.042694] LustreError: 250930:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3132.059261] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3132.062082] Lustre: Skipped 3 previous similar messages [ 3133.383682] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3136.267552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3137.825228] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3138.841610] Lustre: 252071:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3141.100379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3142.246774] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 3143.009104] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3145.733758] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3146.861366] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 3146.861821] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 3147.652524] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3150.654146] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3151.827520] Lustre: 253939:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3153.873568] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 05:58:04 (1785319084) [ 3167.200771] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3167.202766] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3167.206255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3167.207689] Lustre: Skipped 3 previous similar messages [ 3171.804313] Lustre: server umount lustre-MDT0000 complete [ 3172.837997] LustreError: 250930:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3172.845094] LustreError: 250930:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3172.987221] LustreError: 253942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319103 with bad export cookie 11643238605109132619 [ 3172.990180] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3172.991018] LustreError: 253942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3177.956550] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3177.958610] Lustre: Skipped 4 previous similar messages [ 3179.185744] Lustre: server umount lustre-MDT0001 complete [ 3186.555136] Lustre: server umount lustre-OST0000 complete [ 3202.016078] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3202.082713] Lustre: server umount lustre-OST0001 complete [ 3204.189344] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 3207.596773] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3219.471729] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3219.471729] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3219.471729] [ 3219.479802] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3219.560082] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3219.570282] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3219.607799] Lustre: lustre-MDT0000: new disk, initializing [ 3219.629889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3219.632194] Lustre: Skipped 3 previous similar messages [ 3219.637183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3220.949840] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3225.062917] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3225.062917] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3225.062917] [ 3225.080303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3225.117965] Lustre: 257117:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3225.141794] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3225.144921] Lustre: Skipped 1 previous similar message [ 3225.181643] Lustre: lustre-MDT0001: new disk, initializing [ 3225.226808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3225.229764] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3226.605131] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3230.693224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3230.807031] Lustre: lustre-OST0000: new disk, initializing [ 3230.809869] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3230.813779] Lustre: 258699:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3232.570462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3232.573459] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3232.616159] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3232.838153] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3237.069255] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3237.136871] Lustre: lustre-OST0001: new disk, initializing [ 3237.139113] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3237.143810] Lustre: 259570:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3238.272507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3238.275453] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3238.307576] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3239.156291] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3243.411580] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 05:59:33 (1785319173) [ 3246.992699] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 05:59:37 (1785319177) [ 3263.457488] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3263.457544] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3263.460080] LustreError: Skipped 1 previous similar message [ 3263.464935] Lustre: Skipped 5 previous similar messages [ 3263.465625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3263.469663] Lustre: Skipped 1 previous similar message [ 3264.512164] Lustre: server umount lustre-OST0000 complete [ 3269.090148] LustreError: 258684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3269.096921] LustreError: 258684:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3280.352103] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3280.413712] Lustre: server umount lustre-MDT0000 complete [ 3281.579118] LustreError: 257110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319212 with bad export cookie 11643238605109134397 [ 3281.581739] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3281.582313] LustreError: 257110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3287.727748] Lustre: server umount lustre-MDT0001 complete [ 3291.525450] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 3301.406703] Lustre: server umount lustre-OST0001 complete [ 3303.467520] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 3309.933702] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3321.307591] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3321.379210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3321.388311] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3321.420462] Lustre: lustre-MDT0000: new disk, initializing [ 3321.440972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3322.730080] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3326.681938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3326.729594] Lustre: 264096:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3326.833412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3328.105304] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3331.694363] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3333.416735] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3335.390593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3335.495601] Lustre: 266074:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3337.020110] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3337.063645] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3337.426968] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3341.154082] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3347.425590] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3347.427618] LustreError: Skipped 1 previous similar message [ 3347.428234] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3347.429411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3347.431992] Lustre: Skipped 6 previous similar messages [ 3347.433599] Lustre: Skipped 12 previous similar messages [ 3349.523798] Lustre: server umount lustre-OST0000 complete [ 3365.344139] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3365.412757] Lustre: server umount lustre-MDT0000 complete [ 3366.608279] LustreError: 264088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319297 with bad export cookie 11643238605109135867 [ 3366.611093] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3366.611698] LustreError: 264088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3366.718578] Lustre: server umount lustre-MDT0001 complete [ 3369.990457] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 3370.906234] Key type lgssc unregistered [ 3371.018435] LNet: 267833:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3371.021877] LNetError: 267833:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3371.031287] LNet: Removed LNI 192.168.206.129@tcp [ 3371.311134] Key type .llcrypt unregistered [ 3371.312593] Key type ._llcrypt unregistered [ 3380.955467] Key type ._llcrypt registered [ 3380.956813] Key type .llcrypt registered [ 3380.998553] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3381.364201] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3381.422343] alg: No test for adler32 (adler32-zlib) [ 3382.283850] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 3382.375704] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 3383.952117] Key type lgssc registered [ 3384.307232] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3387.022197] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3387.027854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3388.170908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3389.543905] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3392.439604] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3392.562226] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3393.925579] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3396.316279] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3398.126306] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3400.398653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3400.508766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3402.527696] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3404.929149] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3405.276487] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3407.841938] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3407.844300] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3407.847937] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3408.012292] Lustre: server umount lustre-OST0000 complete [ 3411.353139] Lustre: server umount lustre-MDT0000 complete [ 3412.637878] LustreError: 269254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319343 with bad export cookie 16382975983011022360 [ 3412.644434] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3418.808394] Lustre: server umount lustre-MDT0001 complete [ 3422.783560] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 3423.881561] Key type lgssc unregistered [ 3424.008357] LNet: 272852:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3424.010924] LNetError: 272852:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3424.019334] LNet: Removed LNI 192.168.206.129@tcp [ 3424.330610] Key type .llcrypt unregistered [ 3424.331859] Key type ._llcrypt unregistered [ 3430.690309] Key type ._llcrypt registered [ 3430.691137] Key type .llcrypt registered [ 3430.722375] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3431.004081] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3431.042052] alg: No test for adler32 (adler32-zlib) [ 3431.897119] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 3431.999058] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 3433.584136] Key type lgssc registered [ 3433.948926] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3437.091521] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3437.099819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3452.512408] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3457.632263] LustreError: 274023:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.129@tcp: failed processing log, type 4: rc = -110 [ 3484.256166] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3486.065675] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3488.447480] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3489.127758] Lustre: server umount lustre-OST0000 complete [ 3494.031554] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 3494.945889] Key type lgssc unregistered [ 3495.077436] LNet: 275650:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3495.081412] LNetError: 275650:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3495.090273] LNet: Removed LNI 192.168.206.129@tcp [ 3495.399933] Key type .llcrypt unregistered [ 3495.401062] Key type ._llcrypt unregistered [ 3509.236685] Key type ._llcrypt registered [ 3509.237662] Key type .llcrypt registered [ 3509.276360] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3509.723751] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3509.729726] alg: No test for adler32 (adler32-zlib) [ 3510.580828] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 3510.674842] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 3512.264160] Key type lgssc registered [ 3512.627142] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3516.884806] loop0: detected capacity change from 0 to 204800000 [ 3516.995448] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3516.998457] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3517.109709] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3517.469770] loop0: detected capacity change from 0 to 204800000 [ 3517.534100] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3517.537557] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3517.664609] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3518.026496] loop0: detected capacity change from 0 to 204800000 [ 3518.132949] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3518.136076] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3518.177795] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3518.554508] loop0: detected capacity change from 0 to 204800000 [ 3518.627228] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3519.026830] loop0: detected capacity change from 0 to 204800000 [ 3519.039737] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3520.432363] loop0: detected capacity change from 0 to 204800000 [ 3520.445646] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3521.690271] loop0: detected capacity change from 0 to 204800000 [ 3521.700101] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3522.948564] loop0: detected capacity change from 0 to 204800000 [ 3522.963187] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3524.229853] loop0: detected capacity change from 0 to 204800000 [ 3524.252638] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3524.262779] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3524.300120] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3524.304032] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3524.308096] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3524.312485] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3524.317190] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3524.321566] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3524.325458] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3524.328028] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3524.330767] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3524.334279] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3524.338585] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3524.343830] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3524.346929] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3524.349901] Lustre: 279038:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3526.866828] Lustre: server umount MGS complete [ 3527.151431] loop0: detected capacity change from 0 to 204800000 [ 3527.175226] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3527.327129] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3527.340561] Lustre: lustre-MDT0000: Aborting client recovery [ 3527.646173] loop1: detected capacity change from 0 to 204800000 [ 3527.681311] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3527.715781] Lustre: 279468:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3527.720763] Lustre: 279468:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3527.725153] Lustre: 279468:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3527.728142] Lustre: 279468:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3527.730992] Lustre: 279468:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3527.735558] Lustre: 279468:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3527.739386] Lustre: 279468:0:(osd_scrub.c:1864:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3527.746200] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3527.847061] Lustre: lustre-MDT0001: Aborting client recovery [ 3528.151234] loop2: detected capacity change from 0 to 204800000 [ 3528.176399] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3528.215253] Lustre: 279549:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3528.220054] Lustre: 279549:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3528.224811] Lustre: 279549:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3528.228965] Lustre: 279549:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3528.233240] Lustre: 279549:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3528.237274] Lustre: 279549:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3528.240692] Lustre: 279549:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3528.288798] Lustre: Found index 0 for lustre-OST0000, updating log [ 3528.301823] Lustre: 279568:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3528.321112] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3528.324346] Lustre: Skipped 1 previous similar message [ 3528.330373] Lustre: lustre-OST0000: Aborting recovery [ 3528.634388] loop3: detected capacity change from 0 to 204800000 [ 3528.661924] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3528.690342] Lustre: 279633:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3528.692942] Lustre: 279633:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3528.695799] Lustre: 279633:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3528.697965] Lustre: 279633:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3528.700631] Lustre: 279633:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3528.702755] Lustre: 279633:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3528.704996] Lustre: 279633:0:(osd_scrub.c:1864:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3537.890577] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3537.891108] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3537.898878] LustreError: 279412:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 3537.902900] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3537.904483] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3537.905101] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3537.910965] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3539.401281] LustreError: 279378:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 243us: rc = -2 [ 3553.249062] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3553.249395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3553.254104] Lustre: Skipped 2 previous similar messages [ 3553.257267] Lustre: Skipped 3 previous similar messages [ 3556.900545] Lustre: server umount lustre-MDT0000 complete [ 3557.214529] LustreError: 279474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319488 with bad export cookie 11438319171278101228 [ 3557.217063] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3557.219305] LustreError: 279474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3558.368565] LustreError: 279375:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3558.368658] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3558.369111] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3558.376326] LustreError: 279375:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3558.380551] Lustre: Skipped 2 previous similar messages [ 3562.400600] LustreError: 279863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3562.400615] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3562.400623] Lustre: Skipped 1 previous similar message [ 3562.407062] LustreError: 279863:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3563.446379] Lustre: server umount lustre-MDT0001 complete [ 3578.336132] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3578.378828] Lustre: server umount lustre-OST0000 complete [ 3593.184136] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3593.250313] Lustre: server umount lustre-OST0001 complete [ 3596.061326] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:05:26 (1785319526) [ 3605.229904] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 3608.469504] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3619.834854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3619.918253] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3619.933465] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3619.981687] Lustre: lustre-MDT0000: new disk, initializing [ 3620.005382] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3620.007835] Lustre: Skipped 1 previous similar message [ 3620.012801] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3621.411664] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3625.588240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3625.623253] Lustre: 283703:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3625.643110] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3625.645045] Lustre: Skipped 1 previous similar message [ 3625.679192] Lustre: lustre-MDT0001: new disk, initializing [ 3625.709858] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3625.721530] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3625.724965] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3626.986731] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3630.659427] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3632.368518] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3634.420312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3634.509672] Lustre: lustre-OST0000: new disk, initializing [ 3634.511175] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3634.513741] Lustre: 285671:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3634.515568] Lustre: 285671:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 3634.540026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3636.405910] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3640.041710] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3643.900653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3643.904320] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3643.934108] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3647.623378] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3652.789681] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3662.575650] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3672.189236] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3710.433545] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3710.435176] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3710.436748] LustreError: Skipped 1 previous similar message [ 3710.440459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3710.443395] Lustre: Skipped 1 previous similar message [ 3715.554963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3715.557672] Lustre: Skipped 1 previous similar message [ 3724.256127] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3724.309454] Lustre: server umount lustre-OST0000 complete [ 3725.793888] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3725.799084] Lustre: Skipped 1 previous similar message [ 3725.801571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3725.803373] Lustre: Skipped 3 previous similar messages [ 3731.607935] Lustre: server umount lustre-MDT0000 complete [ 3732.725945] LustreError: 287311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319663 with bad export cookie 11438319171278103930 [ 3732.729058] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3732.729971] LustreError: 287311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3732.834457] Lustre: server umount lustre-MDT0001 complete [ 3735.639629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3736.996138] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3738.635457] Lustre: server umount MGS complete [ 3742.350360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3742.508254] LustreError: 289729:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3742.514442] LustreError: 289742:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3742.535303] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3743.838756] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3746.578447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3747.964916] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3750.226887] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3751.949659] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3754.096536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3756.058356] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3758.382613] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3792.352672] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3792.354524] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3792.357279] Lustre: Skipped 1 previous similar message [ 3792.358581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3792.359849] Lustre: Skipped 2 previous similar messages [ 3793.378519] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3795.596218] Lustre: server umount lustre-OST0000 complete [ 3797.984933] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3811.296104] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3811.359244] Lustre: server umount lustre-MDT0000 complete [ 3812.574964] LustreError: 289710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319743 with bad export cookie 11438319171278105841 [ 3812.577821] LustreError: 289710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3812.578693] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3812.696579] Lustre: server umount lustre-MDT0001 complete [ 3816.230670] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 3817.259723] Key type lgssc unregistered [ 3817.381347] LNet: 293699:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3817.383780] LNetError: 293699:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3817.393257] LNet: Removed LNI 192.168.206.129@tcp [ 3817.684228] Key type .llcrypt unregistered [ 3817.685625] Key type ._llcrypt unregistered [ 3829.126690] Key type ._llcrypt registered [ 3829.127684] Key type .llcrypt registered [ 3829.161854] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 3835.177984] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 3835.588101] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3835.596285] alg: No test for adler32 (adler32-zlib) [ 3836.439134] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 3836.518398] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 3838.096135] Key type lgssc registered [ 3838.447898] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3849.755499] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3849.761042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3850.840534] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3850.853235] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3850.887651] Lustre: lustre-MDT0000: new disk, initializing [ 3850.911268] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3850.918330] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3852.281577] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3856.279777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3856.322304] Lustre: 297628:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3856.344541] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3856.346305] Lustre: Skipped 1 previous similar message [ 3856.380338] Lustre: lustre-MDT0001: new disk, initializing [ 3856.409426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3856.420714] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3856.423469] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3857.659907] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3861.265559] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3863.089138] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3865.291385] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3865.395251] Lustre: lustre-OST0000: new disk, initializing [ 3865.397682] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3865.401704] Lustre: 299594:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3865.433630] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3867.405577] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3870.950927] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3872.247088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3872.249940] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3872.274900] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3879.537234] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3883.169540] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3892.819341] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3899.372489] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3933.665656] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3933.666880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3933.667909] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3933.668044] LustreError: Skipped 1 previous similar message [ 3933.672190] Lustre: Skipped 1 previous similar message [ 3938.786866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3938.790746] Lustre: Skipped 2 previous similar messages [ 3939.480682] Lustre: server umount lustre-OST0000 complete [ 3943.392774] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3943.394762] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3943.398168] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3946.841828] Lustre: server umount lustre-MDT0000 complete [ 3948.013324] LustreError: 297663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319878 with bad export cookie 15953536395763710615 [ 3948.015583] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3948.017543] LustreError: 297663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3948.135227] Lustre: server umount lustre-MDT0001 complete [ 3950.905089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3952.314746] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3953.999117] Lustre: server umount MGS complete [ 3957.656173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3957.794848] LustreError: 303609:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3957.817223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3959.028665] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3961.793483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3961.920343] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3963.200563] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3965.448341] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3967.071673] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3969.099719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3970.941266] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 3973.235368] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4007.392615] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4007.394142] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4007.396861] Lustre: Skipped 1 previous similar message [ 4007.398629] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4007.400114] Lustre: Skipped 1 previous similar message [ 4011.474510] Lustre: server umount lustre-OST0000 complete [ 4013.024885] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4013.028683] LustreError: Skipped 1 previous similar message [ 4013.031025] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4013.037153] Lustre: Skipped 1 previous similar message [ 4013.039908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4013.042815] Lustre: Skipped 2 previous similar messages [ 4023.777144] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4023.779568] Lustre: Skipped 4 previous similar messages [ 4027.360177] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4027.427267] Lustre: server umount lustre-MDT0000 complete [ 4028.711214] LustreError: 303579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319959 with bad export cookie 15953536395763712428 [ 4028.713790] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4028.715058] LustreError: 303579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4028.835946] Lustre: server umount lustre-MDT0001 complete [ 4032.381987] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 4033.359437] Key type lgssc unregistered [ 4033.488369] LNet: 307559:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4033.490884] LNetError: 307559:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4033.500319] LNet: Removed LNI 192.168.206.129@tcp [ 4033.816820] Key type .llcrypt unregistered [ 4033.818484] Key type ._llcrypt unregistered [ 4040.489639] Key type ._llcrypt registered [ 4040.490626] Key type .llcrypt registered [ 4040.530300] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 4046.410553] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4046.766231] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4046.832924] alg: No test for adler32 (adler32-zlib) [ 4047.690051] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 4047.784486] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 4049.368131] Key type lgssc registered [ 4049.696656] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4057.282069] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4062.428158] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4062.434577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4063.519519] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4063.533804] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4063.571924] Lustre: lustre-MDT0000: new disk, initializing [ 4063.594748] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4063.601770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4065.070805] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4069.329226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4069.427679] Lustre: lustre-OST0000: new disk, initializing [ 4069.430442] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4069.433139] Lustre: Skipped 1 previous similar message [ 4069.436296] Lustre: 311583:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4069.462476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4069.792520] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4069.795703] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4069.804291] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4071.388545] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4238.817719] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4238.823537] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4242.517253] Lustre: server umount lustre-MDT0000 complete [ 4243.782174] LustreError: 310768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320174 with bad export cookie 11820739432575802724 [ 4243.784443] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4243.785591] LustreError: 310768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4243.850081] Lustre: server umount lustre-OST0000 complete [ 4251.726526] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 4254.749577] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4266.476196] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4266.566228] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4266.577286] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4266.614953] Lustre: lustre-MDT0000: new disk, initializing [ 4266.638492] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4266.645353] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4268.062893] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4272.228198] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4272.267688] Lustre: 315801:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4272.288326] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4272.290811] Lustre: Skipped 1 previous similar message [ 4272.328738] Lustre: lustre-MDT0001: new disk, initializing [ 4272.359681] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4272.371782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4272.377829] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4273.642325] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4277.124704] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4278.824735] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4280.864441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4280.954315] Lustre: lustre-OST0000: new disk, initializing [ 4280.956651] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4280.960064] Lustre: 317769:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4280.987742] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4282.812276] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4283.828912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4283.831791] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4283.859602] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4286.368873] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4287.741028] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4287.811441] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4289.063487] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4289.127109] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4294.114181] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4294.115484] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4294.116172] LustreError: Skipped 1 previous similar message [ 4294.119920] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4294.123587] Lustre: Skipped 1 previous similar message [ 4299.234884] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4299.237360] Lustre: Skipped 1 previous similar message [ 4303.840133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4303.890254] Lustre: server umount lustre-OST0000 complete [ 4308.448928] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4308.452777] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4308.458783] Lustre: Skipped 1 previous similar message [ 4308.461495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4311.207628] Lustre: server umount lustre-MDT0000 complete [ 4312.470325] LustreError: 315794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320243 with bad export cookie 11820739432578743501 [ 4312.473863] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4312.476862] LustreError: 315794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4312.605259] Lustre: server umount lustre-MDT0001 complete [ 4313.776818] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4314.339379] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 06:17:24 (1785320244) [ 4316.532865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4316.684685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4317.899080] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4320.427762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4321.739144] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4323.873588] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4325.482343] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4327.566521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4329.453351] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4331.702264] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4333.900884] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4333.959196] Lustre: lustre-OST0001: new disk, initializing [ 4333.960822] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4333.962925] Lustre: 322914:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4333.989740] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4333.993154] Lustre: Skipped 2 previous similar messages [ 4335.840982] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4339.197416] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4339.201448] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4339.214682] LustreError: 323484:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4339.217928] LustreError: 323484:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4340.258803] LustreError: 323489:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4340.262162] LustreError: 323489:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4341.287586] LustreError: 323489:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4341.291411] LustreError: 323489:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4343.331271] LustreError: 323484:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4343.335634] LustreError: 323484:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4344.562122] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4344.629441] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4346.301347] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9377a0167000.ost_server_uuid 50 [ 4346.778547] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9377a0167000.ost_server_uuid in FULL state after 0 sec [ 4347.426699] LustreError: 323484:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4347.429303] LustreError: 323484:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 4348.490079] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9377a0167000.ost_server_uuid 50 [ 4348.950824] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9377a0167000.ost_server_uuid in FULL state after 0 sec [ 4349.481036] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4361.702341] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4361.705944] Lustre: Skipped 2 previous similar messages [ 4361.706973] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4361.709715] Lustre: Skipped 3 previous similar messages [ 4366.528943] Lustre: server umount lustre-OST0001 complete [ 4366.824909] LustreError: 321899:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4366.829077] LustreError: 321899:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4371.940270] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4371.940954] LustreError: 323485:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4371.941334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4371.943207] LustreError: Skipped 1 previous similar message [ 4371.943485] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4371.943492] Lustre: Skipped 1 previous similar message [ 4371.947446] LustreError: 323485:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4374.091350] Lustre: server umount lustre-OST0000 complete [ 4377.058202] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4381.395814] Lustre: server umount lustre-MDT0000 complete [ 4382.176827] LustreError: 320098:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4382.182109] LustreError: 320098:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4382.537127] LustreError: 322913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320313 with bad export cookie 11820739432578744369 [ 4382.539591] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4382.542031] LustreError: 322913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4382.654732] Lustre: server umount lustre-MDT0001 complete [ 4385.911907] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 4386.841629] Key type lgssc unregistered [ 4386.962304] LNet: 325593:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4386.965248] LNetError: 325593:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4386.975310] LNet: Removed LNI 192.168.206.129@tcp [ 4387.285082] Key type .llcrypt unregistered [ 4387.286457] Key type ._llcrypt unregistered [ 4395.693218] Key type ._llcrypt registered [ 4395.694080] Key type .llcrypt registered [ 4395.727791] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4396.007502] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4396.022345] alg: No test for adler32 (adler32-zlib) [ 4396.864269] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 4396.946944] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 4398.520118] Key type lgssc registered [ 4398.857189] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4401.228441] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4401.232935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4402.359437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4403.579303] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4405.714209] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4407.767954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4407.890772] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4409.142890] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4412.004123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4412.110658] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4413.158150] LustreError: 328861:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4413.161215] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4413.165155] LustreError: 328861:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4413.860080] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4416.120250] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4417.250801] LustreError: 328669:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4417.257362] LustreError: 328669:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4418.272050] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4418.352426] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4420.157341] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4422.412121] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4423.659187] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4424.922113] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4424.988316] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4500.453581] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4500.453657] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4500.455069] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4500.456795] LustreError: Skipped 1 previous similar message [ 4500.461893] Lustre: Skipped 1 previous similar message [ 4505.271357] Lustre: server umount lustre-OST0001 complete [ 4505.570580] LustreError: 328667:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4505.575661] LustreError: 328667:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4509.664983] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4509.668079] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4509.673370] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4509.674818] Lustre: Skipped 1 previous similar message [ 4510.690533] LustreError: 328667:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4510.690607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4510.691714] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4510.691748] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4510.696137] LustreError: 328667:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4510.698021] Lustre: Skipped 1 previous similar message [ 4512.848367] Lustre: server umount lustre-OST0000 complete [ 4515.296665] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4515.298942] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4515.302900] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4516.123715] Lustre: server umount lustre-MDT0000 complete [ 4517.345285] LustreError: 326972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320448 with bad export cookie 11048713465571060186 [ 4517.351989] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4517.484531] Lustre: server umount lustre-MDT0001 complete [ 4520.886782] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 4521.860061] Key type lgssc unregistered [ 4521.980549] LNet: 331919:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4521.983783] LNetError: 331919:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4521.993326] LNet: Removed LNI 192.168.206.129@tcp [ 4522.296533] Key type .llcrypt unregistered [ 4522.297556] Key type ._llcrypt unregistered [ 4539.210138] Key type ._llcrypt registered [ 4539.211110] Key type .llcrypt registered [ 4539.246724] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4539.643653] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4539.650990] alg: No test for adler32 (adler32-zlib) [ 4540.503336] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 4540.580949] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 4542.160121] Key type lgssc registered [ 4542.503377] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4545.332677] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4545.338266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4546.468944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4547.742567] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4550.350022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4550.476404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4551.737638] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4552.682227] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4554.790167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4554.907332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4556.669802] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4556.967896] LustreError: 335670:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4556.972941] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4556.973650] LustreError: 335670:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4559.461456] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4559.535496] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4560.554808] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4561.351169] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4565.299578] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4569.806764] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4576.227077] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4576.227126] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4576.229606] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4576.232427] Lustre: Skipped 2 previous similar messages [ 4576.235825] Lustre: Skipped 3 previous similar messages [ 4579.160497] Lustre: server umount lustre-MDT0000 complete [ 4580.285407] LustreError: 337189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320511 with bad export cookie 17997984208160883193 [ 4580.289298] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4580.289553] LustreError: 337189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4580.395536] Lustre: server umount lustre-MDT0001 complete [ 4591.809181] Lustre: server umount lustre-OST0000 complete [ 4602.208848] Lustre: server umount lustre-OST0001 complete [ 4607.252760] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4610.619933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4610.753576] LustreError: 339443:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4610.771909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4612.034718] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4614.838969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4616.325995] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4617.315548] Lustre: 340585:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4619.434801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4619.551577] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4619.554391] Lustre: Skipped 1 previous similar message [ 4620.580509] LustreError: 340937:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4620.583919] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4621.347055] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4624.071702] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4625.194118] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4625.996603] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4628.996338] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4630.129716] Lustre: 342450:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4635.619459] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4635.622814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4635.622929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4635.623423] Lustre: Skipped 4 previous similar messages [ 4637.210670] Lustre: server umount lustre-MDT0000 complete [ 4638.402063] LustreError: 342453:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320569 with bad export cookie 17997984208160884852 [ 4638.404776] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4638.405771] LustreError: 342453:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4638.513455] Lustre: server umount lustre-MDT0001 complete [ 4649.921120] Lustre: server umount lustre-OST0000 complete [ 4661.334305] Lustre: server umount lustre-OST0001 complete [ 4662.427227] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 4662.895631] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 4663.403062] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 06:23:14 (1785320594) [ 4670.071565] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4672.852524] loop0: detected capacity change from 0 to 1099511627776 [ 4673.029166] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 4673.031830] *** VALIDATE xfs *** [ 4673.035203] XFS (loop0): Mounting V5 Filesystem [ 4673.042497] XFS (loop0): Ending clean mount [ 4673.424629] loop1: detected capacity change from 0 to 18691697672192 [ 4693.376625] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 4696.423103] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4707.308301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4707.389902] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4707.399451] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4707.437297] Lustre: lustre-MDT0000: new disk, initializing [ 4707.456168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4707.458936] Lustre: Skipped 1 previous similar message [ 4707.465115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4708.689299] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4712.590629] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4712.632306] Lustre: 347565:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4712.652314] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4712.653940] Lustre: Skipped 1 previous similar message [ 4712.686647] Lustre: lustre-MDT0001: new disk, initializing [ 4712.728386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4712.732098] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4713.952592] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4717.432744] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4719.101868] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4721.173844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4721.270025] Lustre: lustre-OST0000: new disk, initializing [ 4721.271843] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4721.274045] Lustre: 349543:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4722.872347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4722.875711] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4722.908757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4723.074895] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4726.614772] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4728.016534] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4728.098804] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4729.411541] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4729.481048] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4733.410621] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4733.411412] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4733.415176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4733.416289] LustreError: Skipped 1 previous similar message [ 4733.417877] Lustre: Skipped 3 previous similar messages [ 4736.206602] Lustre: server umount lustre-OST0000 complete [ 4738.529739] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4738.529832] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4738.533839] Lustre: Skipped 1 previous similar message [ 4738.538050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4738.539715] Lustre: Skipped 1 previous similar message [ 4743.509780] Lustre: server umount lustre-MDT0000 complete [ 4743.648892] LustreError: 347574:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4743.654373] LustreError: 347574:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4744.663916] LustreError: 347556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320675 with bad export cookie 17997984208160886385 [ 4744.666578] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4744.666867] LustreError: 347556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4744.774235] Lustre: server umount lustre-MDT0001 complete [ 4747.348731] XFS (loop0): Unmounting Filesystem [ 4748.894193] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 06:24:39 (1785320679) [ 4750.972602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4751.105042] LustreError: 352095:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4751.124995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4751.126852] Lustre: Skipped 2 previous similar messages [ 4752.408346] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4755.125386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4756.534815] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4758.970132] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4760.732942] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4762.927042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4764.894369] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4767.331434] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4781.537426] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4781.537779] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4781.539474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4781.542798] Lustre: Skipped 2 previous similar messages [ 4781.545982] Lustre: Skipped 1 previous similar message [ 4782.921687] Lustre: server umount lustre-OST0000 complete [ 4790.164954] Lustre: server umount lustre-MDT0000 complete [ 4791.314192] LustreError: 352122:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320722 with bad export cookie 17997984208160887267 [ 4791.317248] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4791.317772] LustreError: 352122:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4791.435164] Lustre: server umount lustre-MDT0001 complete [ 4794.739724] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 4795.766219] Key type lgssc unregistered [ 4795.885350] LNet: 355744:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4795.888366] LNetError: 355744:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4795.896391] LNet: Removed LNI 192.168.206.129@tcp [ 4796.207123] Key type .llcrypt unregistered [ 4796.208573] Key type ._llcrypt unregistered [ 4805.599637] Key type ._llcrypt registered [ 4805.600628] Key type .llcrypt registered [ 4805.635880] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 4805.894396] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4805.901909] alg: No test for adler32 (adler32-zlib) [ 4806.741278] Lustre: Lustre: Build Version: 2.17.56_1_g88845ec [ 4806.820629] LNet: Added LNI 192.168.206.129@tcp [8/256/0/180] [ 4808.392084] Key type lgssc registered [ 4808.701869] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4810.947209] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4810.952750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4812.097077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4813.420851] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4815.998631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4816.103311] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4817.312932] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4819.542154] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4821.169949] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4823.249748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4823.353475] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4825.118310] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4827.333967] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4841.952764] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 4841.955070] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4841.959046] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4842.977992] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4842.979754] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4842.980569] Lustre: Skipped 1 previous similar message [ 4846.489037] Lustre: server umount lustre-MDT0001 complete [ 4848.088704] LustreError: 358784:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4848.095779] LustreError: 358784:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4850.074275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4850.110363] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 4850.113339] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4850.128580] Lustre: 357274:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4850.245920] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4851.686289] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 4855.270182] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 4856.806439] LustreError: 361016:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 4856.809218] LustreError: 361016:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 4856.811895] LustreError: 357279:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 4856.830805] LustreError: 361019:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.129@tcp: cfg command failed: rc = -17 [ 4856.834033] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 4856.834033] [ 5155.295576] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.29@tcp (stopping) [ 5157.349255] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5157.349602] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5157.350396] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5157.357139] Lustre: Skipped 1 previous similar message [ 5160.339986] Lustre: server umount lustre-MDT0001 complete [ 5160.408359] LustreError: 357283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5160.413389] LustreError: 357283:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5162.467085] LustreError: 358785:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5162.473414] LustreError: 358785:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5163.489572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5163.526286] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5163.529224] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5163.542752] Lustre: 357276:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5163.647851] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5164.951644] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 5169.126660] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5173.222965] LustreError: 373839:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5173.225227] LustreError: 373839:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5173.227759] LustreError: 357279:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5173.229679] LustreError: 357279:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5173.247545] LustreError: 373842:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 5173.251728] LustreError: 373842:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 5173.254602] LustreError: 373842:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.129@tcp: cfg command failed: rc = -17 [ 5173.257928] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.129@tcp [ 5173.257928] [ 5467.617132] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.29@tcp (stopping) [ 5467.620030] Lustre: Skipped 1 previous similar message [ 5469.153602] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5469.157147] Lustre: Skipped 1 previous similar message [ 5472.915066] Lustre: server umount lustre-MDT0001 complete [ 5474.273585] LustreError: 357288:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5474.277820] LustreError: 357288:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5476.242088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5476.271719] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5476.273885] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5476.287445] Lustre: 357274:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5476.399548] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5477.705936] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 5481.449087] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5484.006965] LustreError: 386449:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5484.009132] LustreError: 386449:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5484.011645] LustreError: 357279:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5484.014906] LustreError: 357279:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5484.032511] LustreError: 386454:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5484.038727] LustreError: 386454:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.129@tcp: cfg command failed: rc = -114 [ 5484.042524] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5484.042524] [ 5793.761270] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5793.763608] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5793.768198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5793.770880] Lustre: Skipped 3 previous similar messages [ 5798.615698] Lustre: server umount lustre-MDT0000 complete [ 5799.856507] LustreError: 357267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321730 with bad export cookie 14046274034684647529 [ 5799.859967] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5799.863477] LustreError: 357267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5801.952909] LustreError: 376657:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5801.953273] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5801.961974] Lustre: Skipped 2 previous similar messages [ 5810.081414] LustreError: 357283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5810.081526] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5810.086837] LustreError: 357283:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5810.088772] Lustre: Skipped 5 previous similar messages [ 5814.240131] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5814.328187] Lustre: server umount lustre-MDT0001 complete [ 5815.660627] Lustre: server umount lustre-OST0000 complete [ 5818.727868] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 5821.796361] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 5833.234568] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 06:42:43 (1785321763) [ 5835.354055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5835.432770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5835.441191] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5835.473174] Lustre: lustre-MDT0000: new disk, initializing [ 5835.491577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5835.496452] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5836.823711] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 5840.878491] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5840.919185] Lustre: 401125:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5840.941918] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5840.944685] Lustre: Skipped 1 previous similar message [ 5840.980288] Lustre: lustre-MDT0001: new disk, initializing [ 5841.010759] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5841.023801] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5841.027718] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5842.262704] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 5845.759753] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5847.488109] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5849.657236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5849.755810] Lustre: lustre-OST0000: new disk, initializing [ 5849.757286] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5849.759714] Lustre: 403095:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5849.790423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5851.627242] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_default_debug -1 all [ 5853.245183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5853.249189] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5853.277284] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5855.171442] Lustre: DEBUG MARKER: oleg629-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5858.274257] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5858.275768] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5858.278596] Lustre: Skipped 2 previous similar messages [ 5858.280319] Lustre: Skipped 1 previous similar message [ 5863.830247] Lustre: server umount lustre-MDT0000 complete [ 5864.991298] LustreError: 401119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321795 with bad export cookie 14046274034684649468 [ 5864.993155] LustreError: 401934:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 5864.994111] LustreError: MGC192.168.206.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5864.995538] LustreError: 401119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5865.112735] Lustre: server umount lustre-MDT0001 complete [ 5876.633299] Lustre: server umount lustre-OST0000 complete [ 5885.552223] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing set_hostid [ 5888.725350] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing load_modules_local [ 5898.766267] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 5793 sec ========== 06:43:49 (1785321829) [ 5899.277200] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 06:43:49 (1785321829) === [ 5900.274339] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 06:43:50 (1785321830) === [ 5912.231458] Lustre: DEBUG MARKER: oleg629-server.virtnet: executing unload_modules_local [ 5913.198643] Key type lgssc unregistered [ 5913.318320] LNet: 409449:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5913.320565] LNetError: 409449:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5913.328398] LNet: Removed LNI 192.168.206.129@tcp [ 5913.617308] Key type .llcrypt unregistered [ 5913.618232] Key type ._llcrypt unregistered