[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 483912239 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2872712K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003071] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005012] kvm-guest: setup PV IPIs [ 0.007580] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010008] pid_max: default: 32768 minimum: 301 [ 0.011154] LSM: Security Framework initializing [ 0.012059] Yama: becoming mindful. [ 0.013037] SELinux: Initializing. [ 0.014068] *** VALIDATE selinux *** [ 0.023174] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028618] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029199] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030113] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031123] *** VALIDATE tmpfs *** [ 0.033260] *** VALIDATE proc *** [ 0.034250] *** VALIDATE cgroup *** [ 0.035010] *** VALIDATE cgroup2 *** [ 0.036295] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037147] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039033] Spectre V2 : User space: Vulnerable [ 0.040008] Speculative Store Bypass: Vulnerable [ 0.043158] debug: unmapping init [mem 0xffffffffb5a59000-0xffffffffb5a60fff] [ 0.046000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046723] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047022] ... version: 2 [ 0.048014] ... bit width: 48 [ 0.049011] ... generic registers: 4 [ 0.050016] ... value mask: 0000ffffffffffff [ 0.051016] ... max period: 00007fffffffffff [ 0.052015] ... fixed-purpose events: 3 [ 0.053012] ... event mask: 000000070000000f [ 0.054331] rcu: Hierarchical SRCU implementation. [ 0.056568] smp: Bringing up secondary CPUs ... [ 0.057590] x86: Booting SMP configuration: [ 0.058030] .... node #0, CPUs: #1 #2 #3 [ 0.061713] smp: Brought up 1 node, 4 CPUs [ 0.063013] smpboot: Max logical packages: 1 [ 0.064015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.132902] node 0 deferred pages initialised in 65ms [ 0.136010] devtmpfs: initialized [ 0.137261] x86/mm: Memory block size: 128MB [ 0.139546] gcov: version magic: 0x41383552 [ 0.141173] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.142092] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.143270] pinctrl core: initialized pinctrl subsystem [ 0.144240] [ 0.144776] ************************************************************* [ 0.145014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146017] ** ** [ 0.147012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.148009] ** ** [ 0.149009] ** This means that this kernel is built to expose internal ** [ 0.150010] ** IOMMU data structures, which may compromise security on ** [ 0.151009] ** your system. ** [ 0.152013] ** ** [ 0.153016] ** If you see this message and you are not debugging the ** [ 0.154016] ** kernel, report this immediately to your vendor! ** [ 0.155014] ** ** [ 0.156014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157017] ************************************************************* [ 0.158805] NET: Registered protocol family 16 [ 0.159449] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.160071] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.161063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.162474] cpuidle: using governor menu [ 0.164219] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.166540] PCI: Using configuration type 1 for base access [ 0.169133] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.179122] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.180034] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.182069] cryptd: max_cpu_qlen set to 1000 [ 0.184273] ACPI: Added _OSI(Module Device) [ 0.185015] ACPI: Added _OSI(Processor Device) [ 0.186015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.187010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.190583] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.193568] ACPI: Interpreter enabled [ 0.194058] ACPI: PM: (supports S0 S3 S4 S5) [ 0.195013] ACPI: Using IOAPIC for interrupt routing [ 0.196131] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.197353] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.206430] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.207041] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.208023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.209083] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.211327] acpiphp: Slot [2] registered [ 0.212126] acpiphp: Slot [5] registered [ 0.213154] acpiphp: Slot [6] registered [ 0.214111] acpiphp: Slot [7] registered [ 0.215146] acpiphp: Slot [8] registered [ 0.216123] acpiphp: Slot [9] registered [ 0.217150] acpiphp: Slot [10] registered [ 0.218138] acpiphp: Slot [3] registered [ 0.219123] acpiphp: Slot [4] registered [ 0.220100] acpiphp: Slot [11] registered [ 0.221091] acpiphp: Slot [12] registered [ 0.222094] acpiphp: Slot [13] registered [ 0.223139] acpiphp: Slot [14] registered [ 0.224091] acpiphp: Slot [15] registered [ 0.225060] acpiphp: Slot [16] registered [ 0.225996] acpiphp: Slot [17] registered [ 0.226049] acpiphp: Slot [18] registered [ 0.226695] acpiphp: Slot [19] registered [ 0.227061] acpiphp: Slot [20] registered [ 0.227849] acpiphp: Slot [21] registered [ 0.228049] acpiphp: Slot [22] registered [ 0.228826] acpiphp: Slot [23] registered [ 0.229071] acpiphp: Slot [24] registered [ 0.229855] acpiphp: Slot [25] registered [ 0.230098] acpiphp: Slot [26] registered [ 0.231091] acpiphp: Slot [27] registered [ 0.232079] acpiphp: Slot [28] registered [ 0.233010] acpiphp: Slot [29] registered [ 0.233962] acpiphp: Slot [30] registered [ 0.234070] acpiphp: Slot [31] registered [ 0.235057] PCI host bridge to bus 0000:00 [ 0.236029] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.237020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.240036] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.242023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.244028] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.247057] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.248198] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.250878] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.255116] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.267010] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.271997] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.276021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.279024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.282018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.284284] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.286615] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.289063] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.291766] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.296014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.306017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.313815] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.318044] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.326000] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.332021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.355022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.364461] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.375022] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.391024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.407017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.416335] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.422024] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.428021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.442021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.455058] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.461020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.465018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.482015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.488777] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.495020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.500016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.511015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.520097] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.524014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.529017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.545025] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.555746] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.558379] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.560389] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.562387] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.565230] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.569204] iommu: Default domain type: Passthrough [ 0.570390] SCSI subsystem initialized [ 0.571202] ACPI: bus type USB registered [ 0.573115] usbcore: registered new interface driver usbfs [ 0.576092] usbcore: registered new interface driver hub [ 0.578139] usbcore: registered new device driver usb [ 0.580191] pps_core: LinuxPPS API ver. 1 registered [ 0.582015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.585134] PTP clock support registered [ 0.588115] EDAC MC: Ver: 3.0.0 [ 0.589126] PCI: Using ACPI for IRQ routing [ 0.591928] NetLabel: Initializing [ 0.594013] NetLabel: domain hash size = 128 [ 0.596026] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.599108] NetLabel: unlabeled traffic allowed by default [ 0.602313] vgaarb: loaded [ 0.604190] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.606017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.613000] clocksource: Switched to clocksource kvm-clock [ 0.728221] VFS: Disk quotas dquot_6.6.0 [ 0.730428] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.734097] *** VALIDATE ramfs *** [ 0.735908] *** VALIDATE hugetlbfs *** [ 0.738785] pnp: PnP ACPI init [ 0.741819] pnp: PnP ACPI: found 6 devices [ 0.757588] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.761517] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.764385] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.767114] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.769932] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.772921] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.776150] NET: Registered protocol family 2 [ 0.778184] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.783263] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.787041] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.792261] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.795787] TCP: Hash tables configured (established 65536 bind 65536) [ 0.798292] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.801204] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.803320] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.806102] NET: Registered protocol family 1 [ 0.808309] RPC: Registered named UNIX socket transport module. [ 0.810029] RPC: Registered udp transport module. [ 0.811243] RPC: Registered tcp transport module. [ 0.812552] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.814952] NET: Registered protocol family 44 [ 0.816710] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.818897] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.821324] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.823747] PCI: CLS 0 bytes, default 64 [ 0.825662] Unpacking initramfs... [ 2.222430] debug: unmapping init [mem 0xffff9e3f3cc54000-0xffff9e3f3ffbffff] [ 2.226571] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.228899] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.231923] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.695260] Initialise system trusted keyrings [ 2.697666] Key type blacklist registered [ 2.700083] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.708961] zbud: loaded [ 2.712299] *** VALIDATE nfs *** [ 2.713611] *** VALIDATE nfs4 *** [ 2.715165] pstore: using deflate compression [ 2.718961] Platform Keyring initialized [ 2.818831] NET: Registered protocol family 38 [ 2.820272] Key type asymmetric registered [ 2.821586] Asymmetric key parser 'x509' registered [ 2.823118] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.825884] io scheduler mq-deadline registered [ 2.827617] io scheduler kyber registered [ 2.829092] io scheduler bfq registered [ 2.831078] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.833985] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.836691] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.839663] ACPI: Power Button [PWRF] [ 2.843731] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.848379] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.858742] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.866205] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.877444] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.905838] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.933847] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.939160] Non-volatile memory driver v1.3 [ 2.941582] Linux agpgart interface v0.103 [ 2.972643] virtio_blk virtio1: [vda] 150672 512-byte logical blocks (77.1 MB/73.6 MiB) [ 2.975567] vda: detected capacity change from 0 to 77144064 [ 2.991813] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.994637] vdb: detected capacity change from 0 to 1073741824 [ 3.007845] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.010412] vdc: detected capacity change from 0 to 2621440000 [ 3.025737] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.027731] vdd: detected capacity change from 0 to 2621440000 [ 3.038587] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.040775] vde: detected capacity change from 0 to 4294967296 [ 3.052606] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.055746] vdf: detected capacity change from 0 to 4294967296 [ 3.063487] libphy: Fixed MDIO Bus: probed [ 3.073570] usbcore: registered new interface driver usbserial_generic [ 3.075966] usbserial: USB Serial support registered for generic [ 3.078517] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.083636] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.085423] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.087947] mousedev: PS/2 mouse device common for all mice [ 3.092728] rtc_cmos 00:05: RTC can wake from S4 [ 3.095657] rtc_cmos 00:05: registered as rtc0 [ 3.097344] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.100209] intel_pstate: CPU model not supported [ 3.102267] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.105746] hid: raw HID events driver (C) Jiri Kosina [ 3.105863] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.107120] usbcore: registered new interface driver usbhid [ 3.110171] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.110705] usbhid: USB HID core driver [ 3.110834] drop_monitor: Initializing network drop monitor service [ 3.115509] Initializing XFRM netlink socket [ 3.116843] NET: Registered protocol family 10 [ 3.119599] Segment Routing with IPv6 [ 3.121060] NET: Registered protocol family 17 [ 3.122996] mpls_gso: MPLS GSO support [ 3.127182] RAS: Correctable Errors collector initialized. [ 3.128929] AVX version of gcm_enc/dec engaged. [ 3.130573] AES CTR mode by8 optimization enabled [ 3.191554] sched_clock: Marking stable (3191523973, 0)->(4136639924, -945115951) [ 3.195278] registered taskstats version 1 [ 3.197191] Loading compiled-in X.509 certificates [ 3.198638] zswap: loaded using pool lzo/zbud [ 3.215453] Key type big_key registered [ 3.224428] Key type encrypted registered [ 3.225676] ima: No TPM chip found, activating TPM-bypass! [ 3.227312] ima: Allocated hash algorithm: sha1 [ 3.229337] ima: No architecture policies found [ 3.230802] evm: Initialising EVM extended attributes: [ 3.232262] evm: security.selinux [ 3.233213] evm: security.ima [ 3.234051] evm: security.capability [ 3.235070] evm: HMAC attrs: 0x1 [ 3.236789] rtc_cmos 00:05: setting system clock to 2026-09-30 12:36:35 UTC (1790771795) [ 3.242339] debug: unmapping init [mem 0xffffffffb6a03000-0xffffffffb6bfffff] [ 3.245186] debug: unmapping init [mem 0xffffffffb5782000-0xffffffffb5a58fff] [ 3.257124] Write protecting the kernel read-only data: 28672k [ 3.261131] debug: unmapping init [mem 0xffffffffb3e03000-0xffffffffb3ffffff] [ 3.264535] debug: unmapping init [mem 0xffffffffb4714000-0xffffffffb47fffff] [ 3.289663] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.296686] systemd[1]: Detected virtualization kvm. [ 3.298903] systemd[1]: Detected architecture x86-64. [ 3.301094] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.330160] systemd[1]: No hostname configured. [ 3.332013] systemd[1]: Set hostname to . [ 3.333941] random: systemd: uninitialized urandom read (16 bytes read) [ 3.336484] systemd[1]: Initializing machine ID from random generator. [ 3.380041] random: ln: uninitialized urandom read (6 bytes read) [ 3.452247] random: systemd: uninitialized urandom read (16 bytes read) [ 3.454741] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.457908] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.462253] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.959294] device-mapper: uevent: version 1.0.3 [ 3.961524] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.695769] virtio_net virtio0 ens2: renamed from eth0 [ 4.716713] random: fast init done [ 4.739275] scsi host0: ata_piix [ 4.765364] scsi host1: ata_piix [ 4.766983] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.769745] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.355266] dracut-initqueue[583]: RTNETLINK answers: File exists [ 9.746579] random: crng init done [ 9.749531] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.194418] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.431105] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.682724] SELinux: Disabled at runtime. [ 11.751127] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.761210] systemd[1]: Detected virtualization kvm. [ 11.763177] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.236284] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.239473] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.243755] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.247700] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.251494] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.259795] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.264180] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on Process Core Dump Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. Mounting Huge Pages File System... Starting Apply Kernel Variables... [ 12.334875] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Starting Remount Root and Kernel File Systems... Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.738774] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.966844] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.030880] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.171315] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.181981] EDAC sbridge: Ver: 1.1.2 [ 14.718536] Key type dns_resolver registered [ 15.018867] NFS: Registering the id_resolver key type [ 15.020744] Key type id_resolver registered [ 15.022163] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg238-server login: [ 32.211118] hrtimer: interrupt took 3308567 ns [ 61.669216] libcfs: loading out-of-tree module taints kernel. [ 61.721721] Key type ._llcrypt registered [ 61.727820] Key type .llcrypt registered [ 61.866149] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 82.134993] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 83.697393] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 83.717722] alg: No test for adler32 (adler32-zlib) [ 85.156551] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 86.193057] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 87.959872] Key type lgssc registered [ 90.106262] Lustre: Echo OBD driver; http://www.lustre.org/ [ 109.637609] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 152.044895] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 164.424992] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 164.473757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 165.789248] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 165.820619] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 165.931413] Lustre: lustre-MDT0000: new disk, initializing [ 165.997945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 166.008697] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 169.920853] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 182.560823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 182.634879] Lustre: 6505:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 182.657798] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 182.662413] Lustre: Skipped 1 previous similar message [ 182.750347] Lustre: lustre-MDT0001: new disk, initializing [ 182.814578] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 182.868029] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 182.883853] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 186.840658] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 192.160833] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 205.278749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 205.701428] Lustre: lustre-OST0000: new disk, initializing [ 205.706409] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 205.716040] Lustre: 8443:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 205.879442] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 207.288134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 207.308198] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 207.508920] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 213.292346] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 230.104435] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 230.287717] Lustre: lustre-OST0001: new disk, initializing [ 230.298606] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 230.313742] Lustre: 9516:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 230.410692] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 238.103135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 238.113354] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 238.177332] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 238.918986] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 253.154502] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 261.186426] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 268.309384] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing check_logdir /tmp/testlogs/ [ 273.178934] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing yml_node [ 278.231426] Lustre: DEBUG MARKER: Client: 2.17.59.3 [ 280.861725] Lustre: DEBUG MARKER: MDS: 2.17.59.3 [ 283.363256] Lustre: DEBUG MARKER: OSS: 2.17.59.3 [ 284.893228] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Sep 30 08:41:16 EDT 2026 [ 302.927162] Lustre: DEBUG MARKER: only running tests: 76-120 [ 304.535671] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 306.102481] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 325.089899] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 325.091037] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 325.121955] Lustre: Skipped 3 previous similar messages [ 325.142419] Lustre: Skipped 3 previous similar messages [ 330.213855] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 330.219166] Lustre: Skipped 3 previous similar messages [ 335.330451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 335.343604] Lustre: Skipped 3 previous similar messages [ 336.353057] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 336.540399] Lustre: server umount lustre-MDT0000 complete [ 339.948814] LustreError: 6497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790772132 with bad export cookie 8656643421999409386 [ 339.956542] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 340.450902] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 340.462717] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 340.471423] Lustre: Skipped 1 previous similar message [ 340.484240] Lustre: Skipped 1 previous similar message [ 344.485395] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 344.502176] Lustre: Skipped 1 previous similar message [ 346.436240] Lustre: server umount lustre-MDT0001 complete [ 350.279853] Lustre: server umount lustre-OST0000 complete [ 354.209909] Lustre: server umount lustre-OST0001 complete [ 361.022885] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 370.738846] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 413.698403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 414.123605] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 414.167079] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 414.267255] Lustre: lustre-MDT0000: new disk, initializing [ 414.348535] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 414.373751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 419.660845] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 434.275116] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 434.570730] Lustre: 15118:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 434.701667] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 434.704646] Lustre: Skipped 1 previous similar message [ 434.821945] Lustre: lustre-MDT0001: new disk, initializing [ 434.956648] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 435.039200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 435.054960] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 440.733423] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 453.417216] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 459.789733] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 466.091193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 466.425310] Lustre: lustre-OST0000: new disk, initializing [ 466.429436] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 466.435522] Lustre: 17090:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 466.541487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 468.295028] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 468.304773] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 468.420257] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 473.338797] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 484.033877] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 489.258822] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 489.474239] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 495.792136] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 496.040802] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 499.173145] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 499.174737] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 499.179060] LustreError: Skipped 1 previous similar message [ 499.180735] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 499.206475] Lustre: Skipped 1 previous similar message [ 504.567377] Lustre: server umount lustre-OST0000 complete [ 509.417415] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 509.439840] Lustre: Skipped 1 previous similar message [ 515.223749] Lustre: server umount lustre-MDT0000 complete [ 519.651414] LustreError: 15130:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.662328] LustreError: 15130:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 520.055086] LustreError: 15110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790772312 with bad export cookie 8656643421999410926 [ 520.065306] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 520.072558] LustreError: 15110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 520.844283] Lustre: server umount lustre-MDT0001 complete [ 527.700266] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:45:18 (1790772318) [ 535.509101] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 535.845848] LustreError: 19357:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 535.913968] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 540.744328] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 541.155598] LustreError: 19358:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 546.284659] LustreError: 19357:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 549.950647] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 554.561202] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 563.142290] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 570.085292] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 578.507855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 585.369570] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 593.482270] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 600.395903] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 610.799449] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 610.816601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 610.817198] Lustre: Skipped 1 previous similar message [ 610.838485] Lustre: Skipped 9 previous similar messages [ 614.672198] Lustre: server umount lustre-MDT0000 complete [ 615.905383] LustreError: 19358:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 615.921975] LustreError: 19358:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 618.036595] LustreError: 19339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790772410 with bad export cookie 8656643421999411808 [ 618.046697] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 618.049906] LustreError: 19339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 618.425400] Lustre: server umount lustre-MDT0001 complete [ 632.750847] Lustre: server umount lustre-OST0000 complete [ 654.999641] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 665.548673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 665.934092] LustreError: 23740:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 666.017228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 666.022026] Lustre: Skipped 2 previous similar messages [ 670.837914] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 679.236157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 684.205780] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 687.351458] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 693.586729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 701.279528] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 709.927982] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 710.187722] Lustre: lustre-OST0001: new disk, initializing [ 710.191938] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 710.199423] Lustre: 26053:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 711.736499] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 711.746594] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 711.907669] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 716.967027] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 726.726121] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 730.362921] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 742.885791] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 742.889125] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 742.913285] Lustre: Skipped 3 previous similar messages [ 742.928172] Lustre: Skipped 1 previous similar message [ 747.630965] Lustre: server umount lustre-MDT0000 complete [ 748.015769] LustreError: 25253:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 748.065916] LustreError: 25253:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 751.296185] LustreError: 26818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790772543 with bad export cookie 8656643421999412655 [ 751.306121] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 751.312285] LustreError: 26818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 751.794358] Lustre: server umount lustre-MDT0001 complete [ 765.543684] Lustre: server umount lustre-OST0000 complete [ 779.704960] Lustre: server umount lustre-OST0001 complete [ 796.264816] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 807.750728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 808.039043] LustreError: 29075:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 808.050216] LustreError: 29075:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 808.141053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 808.148659] Lustre: Skipped 3 previous similar messages [ 813.020922] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 824.347914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 829.902793] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 833.429550] Lustre: 30223:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 841.333726] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 849.013995] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 857.895753] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 864.941980] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 873.253968] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 876.619792] Lustre: 32101:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 883.704161] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 883.713420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 883.728730] Lustre: Skipped 4 previous similar messages [ 883.740335] Lustre: Skipped 5 previous similar messages [ 889.582281] Lustre: server umount lustre-MDT0000 complete [ 893.356867] LustreError: 29055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790772685 with bad export cookie 8656643421999413950 [ 893.366447] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 893.366944] LustreError: 29055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 893.739169] Lustre: server umount lustre-MDT0001 complete [ 908.667263] Lustre: server umount lustre-OST0000 complete [ 910.309281] Lustre: 3635:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790772686/real 1790772686] req@ffff9e3fbd450a80 x1877760413048576/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790772702 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 912.965655] Lustre: server umount lustre-OST0001 complete [ 923.727475] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:51:54 (1790772714) [ 954.916849] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 964.631944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 965.125446] LustreError: 34907:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 965.141458] LustreError: 34907:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 969.659089] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 978.187456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 984.804601] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 988.309093] Lustre: 36062:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 995.682613] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1004.100809] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1015.557975] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1023.439332] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1033.319595] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1037.864022] Lustre: 37940:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1045.473140] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1045.477541] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1045.490098] Lustre: Skipped 1 previous similar message [ 1045.494357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1045.498586] Lustre: Skipped 4 previous similar messages [ 1050.519073] Lustre: server umount lustre-MDT0000 complete [ 1055.290242] LustreError: 34887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790772847 with bad export cookie 8656643421999415056 [ 1055.303065] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1055.309352] LustreError: 34887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1055.895815] Lustre: server umount lustre-MDT0001 complete [ 1070.231513] Lustre: server umount lustre-OST0000 complete [ 1073.121653] Lustre: 3637:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790772849/real 1790772849] req@ffff9e3fbe96a680 x1877760413101824/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790772865 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1075.941661] Lustre: server umount lustre-OST0001 complete [ 1090.335480] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:54:41 (1790772881) [ 1108.715080] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1120.833897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1121.227866] LustreError: 40083:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1121.238577] LustreError: 40083:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 12 previous similar messages [ 1121.309905] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1121.316752] Lustre: Skipped 7 previous similar messages [ 1125.536418] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1134.492536] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1140.182818] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1143.515174] Lustre: 41235:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1150.388952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1157.878846] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1167.301420] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1173.912654] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1182.385548] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1186.143518] Lustre: 43110:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1186.959811] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1186.974077] Lustre: Skipped 1 previous similar message [ 1191.399360] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1191.411174] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1191.421440] Lustre: Skipped 4 previous similar messages [ 1196.184318] Lustre: server umount lustre-MDT0000 complete [ 1201.106687] LustreError: 40064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790772993 with bad export cookie 8656643421999416162 [ 1201.111561] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1201.129212] LustreError: 40064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1207.687823] Lustre: server umount lustre-MDT0001 complete [ 1216.082790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1226.209928] LustreError: 3634:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9e3fbe93ea00 x1877760413154304/t0(0) o250->MGC192.168.202.138@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1234.550789] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1236.396611] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1245.214982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1251.106539] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1251.318327] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1251.331651] Lustre: Skipped 1 previous similar message [ 1258.680216] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1260.171780] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1264.965293] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1266.562643] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1276.387574] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1278.090909] Lustre: server umount lustre-MDT0000 complete [ 1281.502286] LustreError: 40064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790773073 with bad export cookie 8656643421999417289 [ 1281.505958] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1281.517801] LustreError: 40064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1281.946674] Lustre: server umount lustre-MDT0001 complete [ 1297.349865] Lustre: server umount lustre-OST0000 complete [ 1298.208240] Lustre: 3637:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790773074/real 1790773074] req@ffff9e3fa015e300 x1877760413188992/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790773090 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1302.049533] Lustre: server umount lustre-OST0001 complete [ 1312.423863] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 08:58:23 (1790773103) [ 1329.029231] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1339.265334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1344.379442] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1353.558597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1359.210312] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1362.207617] Lustre: 48870:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1368.186109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1374.926888] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1379.760443] LustreError: 49224:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1379.781750] LustreError: 49224:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 35 previous similar messages [ 1383.759832] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1383.855798] Lustre: lustre-OST0001: Not available for connect from 0@lo (not set up) [ 1383.863920] Lustre: Skipped 22 previous similar messages [ 1389.908272] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1397.179847] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1402.384276] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1420.259976] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1420.277818] Lustre: Skipped 12 previous similar messages [ 1424.675667] Lustre: server umount lustre-MDT0000 complete [ 1429.652052] LustreError: 47700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790773221 with bad export cookie 8656643421999418094 [ 1429.661109] LustreError: 47700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1429.664226] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1430.113436] Lustre: server umount lustre-MDT0001 complete [ 1445.769968] Lustre: server umount lustre-OST0000 complete [ 1446.625549] Lustre: 3636:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790773222/real 1790773222] req@ffff9e3e86ecc000 x1877760413246976/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790773238 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1450.857200] Lustre: server umount lustre-OST0001 complete [ 1462.954842] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:00:53 (1790773253) [ 1465.589132] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1467.932185] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:00:58 (1790773258) [ 1477.586521] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 1487.623845] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1511.422928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1511.836424] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1511.929391] Lustre: lustre-MDT0000: new disk, initializing [ 1512.011389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1515.955794] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1525.647030] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1532.511415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1532.741270] Lustre: lustre-OST0000: new disk, initializing [ 1532.744511] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1532.747936] Lustre: Skipped 1 previous similar message [ 1532.752039] Lustre: 54959:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1534.434828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1534.451154] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1534.509392] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1538.402727] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1547.708748] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1549.876794] Lustre: DEBUG MARKER: create test files [ 1585.647643] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1588.789354] Lustre: server umount lustre-OST0000 complete [ 1592.756521] Lustre: server umount lustre-MDT0000 complete [ 1604.902560] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 1607.414725] Key type lgssc unregistered [ 1607.736631] LNet: 56587:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1607.747465] LNetError: 56587:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1608.810248] LNet: Removed LNI 192.168.202.138@tcp [ 1609.731329] Key type .llcrypt unregistered [ 1609.733705] Key type ._llcrypt unregistered [ 1642.478516] Key type ._llcrypt registered [ 1642.481311] Key type .llcrypt registered [ 1642.672469] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1643.999699] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1644.040033] alg: No test for adler32 (adler32-zlib) [ 1645.283941] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 1645.590608] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 1647.375228] Key type lgssc registered [ 1648.655686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1657.471454] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1657.502884] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1659.039564] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1663.333457] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1670.628667] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1678.703795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1679.248122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1684.351959] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1684.542866] Lustre: 59341:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1684.551776] Lustre: 59341:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1684.557655] Lustre: 59341:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1684.563772] Lustre: 59341:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1684.581249] Lustre: 59341:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1684.591298] Lustre: 59341:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1686.541501] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1694.342259] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1701.021964] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1706.640842] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1715.174815] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1715.179869] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1715.211106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1716.127604] Lustre: server umount lustre-OST0000 complete [ 1720.255832] Lustre: server umount lustre-MDT0000 complete [ 1730.876142] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 1733.766116] Key type lgssc unregistered [ 1734.085572] LNet: 60945:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1734.090237] LNetError: 60945:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1734.109241] LNet: Removed LNI 192.168.202.138@tcp [ 1734.949033] Key type .llcrypt unregistered [ 1734.952596] Key type ._llcrypt unregistered [ 1766.179716] Key type ._llcrypt registered [ 1766.181917] Key type .llcrypt registered [ 1766.304627] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1767.845135] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1767.859701] alg: No test for adler32 (adler32-zlib) [ 1769.001526] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 1769.250796] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 1770.975227] Key type lgssc registered [ 1772.138353] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1779.279812] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1779.296246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1780.576291] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1784.758566] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1791.855427] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1798.322495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1798.697184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1804.828089] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1804.866952] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1804.946190] Lustre: 63795:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1804.954993] Lustre: 63795:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1804.961870] Lustre: 63795:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1804.974751] Lustre: 63795:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1804.980522] Lustre: 63795:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1804.994042] Lustre: 63795:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1813.934156] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1819.722652] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1861.097926] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1861.105857] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1861.115506] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1864.319869] Lustre: server umount lustre-OST0000 complete [ 1867.417940] Lustre: server umount lustre-MDT0000 complete [ 1879.731317] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 1883.095927] Key type lgssc unregistered [ 1883.440756] LNet: 65378:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1883.451671] LNetError: 65378:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1883.483409] LNet: Removed LNI 192.168.202.138@tcp [ 1884.237964] Key type .llcrypt unregistered [ 1884.244811] Key type ._llcrypt unregistered [ 1905.663889] Key type ._llcrypt registered [ 1905.666643] Key type .llcrypt registered [ 1905.768286] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 1923.415600] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1924.342889] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1924.543647] alg: No test for adler32 (adler32-zlib) [ 1925.645768] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 1925.934789] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 1927.623246] Key type lgssc registered [ 1928.742707] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1969.961797] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1970.006434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1971.233239] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1971.262454] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1971.356558] Lustre: lustre-MDT0000: new disk, initializing [ 1971.437590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1971.455220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1976.064629] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1987.908379] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1988.047916] Lustre: 68547:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1988.092381] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1988.095730] Lustre: Skipped 1 previous similar message [ 1988.165517] Lustre: lustre-MDT0001: new disk, initializing [ 1988.250445] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1988.274454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1988.287633] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1993.886324] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2003.945512] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2011.054155] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2018.722750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2018.990143] Lustre: lustre-OST0000: new disk, initializing [ 2018.995168] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2019.005397] Lustre: 70515:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2019.125088] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2026.630417] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2028.141888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2028.161640] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2028.333217] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2036.909899] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2041.974850] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2042.209420] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2047.420176] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2047.574615] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2054.112527] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2054.115649] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2054.129426] LustreError: Skipped 1 previous similar message [ 2054.137207] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2054.151943] Lustre: Skipped 1 previous similar message [ 2054.173437] Lustre: Skipped 1 previous similar message [ 2059.240038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2059.251557] Lustre: Skipped 1 previous similar message [ 2064.353115] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2064.355799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2064.386132] Lustre: Skipped 1 previous similar message [ 2064.750512] Lustre: server umount lustre-OST0000 complete [ 2069.475830] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2069.501358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2074.592961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2074.597061] Lustre: Skipped 2 previous similar messages [ 2083.295894] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2083.625548] Lustre: server umount lustre-MDT0000 complete [ 2087.597367] LustreError: 68539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790773879 with bad export cookie 18283713087051334854 [ 2087.601130] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2087.614387] LustreError: 68539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2087.951082] Lustre: server umount lustre-MDT0001 complete [ 2097.061252] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:11:28 (1790773888) [ 2105.911337] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2144.705746] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 2153.944602] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2193.969951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2194.241789] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2194.289775] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2194.397840] Lustre: lustre-MDT0000: new disk, initializing [ 2194.485632] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2194.497784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2198.399840] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2208.709698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2208.875553] Lustre: 76538:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2208.932935] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2208.936283] Lustre: Skipped 1 previous similar message [ 2209.016143] Lustre: lustre-MDT0001: new disk, initializing [ 2209.131253] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2209.168362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2209.185601] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2212.655489] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2225.735702] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2233.165207] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2246.088697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2246.281587] Lustre: lustre-OST0000: new disk, initializing [ 2246.286270] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2246.290378] Lustre: 78508:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2246.354585] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2252.360388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2252.371873] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2252.523488] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2252.922802] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2262.076366] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2267.279660] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2267.647595] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2272.692510] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2272.920308] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2278.375892] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2278.376563] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2278.385330] LustreError: Skipped 1 previous similar message [ 2278.391353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2278.410423] Lustre: Skipped 2 previous similar messages [ 2278.414605] Lustre: Skipped 2 previous similar messages [ 2281.682798] Lustre: server umount lustre-OST0000 complete [ 2286.050271] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2286.052909] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2297.825984] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2297.835838] Lustre: Skipped 7 previous similar messages [ 2300.383496] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2300.673078] Lustre: server umount lustre-MDT0000 complete [ 2302.952431] LustreError: 77417:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2302.990517] LustreError: 77417:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2304.204475] LustreError: 76532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790774096 with bad export cookie 18283713087051335736 [ 2304.207613] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2304.220939] LustreError: 76532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2304.656836] Lustre: server umount lustre-MDT0001 complete [ 2314.274315] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:15:05 (1790774105) [ 2322.307712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2322.806406] LustreError: 80922:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2322.918836] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2327.704804] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2328.033107] LustreError: 80923:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2333.167712] LustreError: 80922:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2338.282186] LustreError: 80923:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2338.571802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2343.474441] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2351.793953] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2359.362194] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2368.395074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2368.969672] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2368.972761] Lustre: Skipped 1 previous similar message [ 2376.963967] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2385.716505] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2390.647071] Lustre: 83596:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting 52be53a3-887e-4eee-9f98-d356a5d1690e at adminstrative request [ 2393.581934] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2393.618736] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2402.796484] Lustre: MGS: Client 52be53a3-887e-4eee-9f98-d356a5d1690e (at 0@lo) reconnecting [ 2402.810339] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xfdbccc2e70db0baa to 0xfdbccc2e70db0e4a [ 2402.822386] Lustre: MGC192.168.202.138@tcp: Connection restored to 0@lo (at 0@lo) [ 2428.793510] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2428.945703] Lustre: lustre-OST0001: new disk, initializing [ 2428.948483] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2428.953183] Lustre: 83988:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2434.825559] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2438.727674] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2438.742118] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2438.896264] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2445.007441] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2449.376323] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2449.377304] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2449.392167] LustreError: Skipped 1 previous similar message [ 2449.392790] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2449.435441] Lustre: Skipped 2 previous similar messages [ 2454.459465] Lustre: server umount lustre-OST0001 complete [ 2454.511741] LustreError: 83630:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2454.527834] LustreError: 83630:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2460.128894] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2460.138781] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2466.477892] Lustre: server umount lustre-OST0000 complete [ 2471.915164] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2471.922292] LustreError: Skipped 1 previous similar message [ 2477.756602] Lustre: server umount lustre-MDT0000 complete [ 2480.097516] LustreError: 80919:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2480.129371] LustreError: 80919:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 2481.910337] LustreError: 80904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790774274 with bad export cookie 18283713087051337290 [ 2481.912719] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2481.919323] LustreError: 80904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2482.303872] Lustre: server umount lustre-MDT0001 complete [ 2491.190632] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:18:02 (1790774282) [ 2492.548626] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2494.205194] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:18:05 (1790774285) [ 2495.695689] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2497.541494] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:18:08 (1790774288) [ 2499.541559] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2501.987973] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:18:12 (1790774292) [ 2504.262898] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2506.334555] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:18:17 (1790774297) [ 2508.228742] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2514.141864] loop: module loaded [ 2514.155110] loop0: detected capacity change from 0 to 4294967296 [ 2514.213162] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2515.636685] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2518.874710] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2522.526525] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2549.486867] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2549.524542] LustreError: 88120:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2549.535870] LustreError: 88120:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2549.548385] LustreError: 88120:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2549.558083] LustreError: 88120:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2570.048772] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 2581.056930] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2624.468807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2624.746430] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2624.777921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2624.880509] Lustre: lustre-MDT0000: new disk, initializing [ 2624.983276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2624.991784] Lustre: Skipped 1 previous similar message [ 2625.005932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2629.636632] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2640.662701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2640.848079] Lustre: 90984:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2641.140884] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2645.941883] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2655.555660] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2661.441691] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2667.440774] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2667.656539] Lustre: 92955:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2673.858846] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2675.792195] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2675.880401] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2682.174636] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2686.269421] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2686.419163] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2692.613281] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2692.974431] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2696.692410] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2696.699495] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2696.727704] Lustre: Skipped 3 previous similar messages [ 2696.738623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2696.739941] LustreError: Skipped 1 previous similar message [ 2696.762386] Lustre: Skipped 9 previous similar messages [ 2701.606761] Lustre: server umount lustre-OST0000 complete [ 2711.806101] Lustre: server umount lustre-MDT0000 complete [ 2712.033510] LustreError: 90992:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2712.055484] LustreError: 90992:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2715.234345] LustreError: 90978:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790774507 with bad export cookie 18283713087051337906 [ 2715.242873] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2715.251899] LustreError: 90978:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2715.585873] Lustre: server umount lustre-MDT0001 complete [ 2724.277634] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:21:55 (1790774515) [ 2731.920735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2736.878616] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2746.180866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2751.962772] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2760.225287] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2766.586405] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2773.922214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2774.367701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2774.381702] Lustre: Skipped 4 previous similar messages [ 2781.268230] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2789.839470] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2795.198047] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2795.424736] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2800.641977] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2800.875427] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2808.887287] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2809.111785] Lustre: lustre-OST0001: new disk, initializing [ 2809.116899] Lustre: Skipped 2 previous similar messages [ 2809.124305] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2809.130506] Lustre: Skipped 3 previous similar messages [ 2809.134782] Lustre: 98742:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2815.671190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2815.685511] Lustre: Skipped 2 previous similar messages [ 2815.700168] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2815.816540] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2816.399492] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2827.069985] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2832.748197] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2833.056935] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2838.553922] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2838.975681] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2861.284502] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2887.305607] Lustre: Failing over lustre-MDT0000 [ 2887.652464] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2887.665292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2887.669938] Lustre: Skipped 5 previous similar messages [ 2887.684202] Lustre: Skipped 5 previous similar messages [ 2887.797438] Lustre: server umount lustre-MDT0000 complete [ 2887.954116] LustreError: 95365:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2887.975726] LustreError: 95365:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2909.670914] Lustre: 66759:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790774685/real 1790774685] req@0000000051a234b9 x1877762342411520/t0(0) o400->MGC192.168.202.138@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790774701 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2909.712994] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2913.620460] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2913.622354] LDISKFS-fs (dm-0): recovery complete [ 2913.635451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2918.883155] LustreError: 66757:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000010512b42 x1877762342419456/t0(0) o250->MGC192.168.202.138@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2919.370301] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2919.657560] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2919.662530] Lustre: 95365:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2924.172555] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2924.517808] Lustre: 95369:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2924.528133] Lustre: 95369:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2924.544866] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2924.648305] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2924.967086] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2925.312133] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2925.327243] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2925.648880] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2925.664304] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2925.675543] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2925.685312] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2926.332034] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2926.345951] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2926.679161] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2926.698850] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2927.986267] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2927.992746] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2928.625814] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2928.637045] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2928.968040] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2928.986451] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2932.169549] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2932.182426] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2932.834611] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2932.857333] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2933.191202] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2933.196557] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2940.191739] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2940.222054] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 2940.869848] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2940.897058] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2941.223209] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2941.233514] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2956.198897] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2956.214775] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 49 previous similar messages [ 2957.168431] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2957.185610] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2957.511138] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2957.517625] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2979.295303] Lustre: 101589:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2979.314901] Lustre: 101589:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2979.321219] Lustre: 101589:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 2708e39d-78e2-4fea-97ca-b98a1a8959e9@192.168.202.38@tcp [ 2979.329393] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2979.332927] Lustre: 101589:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@000000002bcb1858 x1877762324367616/t0(8589934767) o101->2708e39d-78e2-4fea-97ca-b98a1a8959e9@192.168.202.38@tcp:97/0 lens 592/0 e 0 to 0 dl 1790774782 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2979.357966] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2979.378810] Lustre: lustre-MDT0000: Denying connection for new client 2708e39d-78e2-4fea-97ca-b98a1a8959e9 (at 192.168.202.38@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2979.391988] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2979.420322] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2979.423690] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2979.434102] Lustre: Skipped 2 previous similar messages [ 2979.472073] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 2979.474853] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2995.680938] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2995.682754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2995.697197] LustreError: Skipped 1 previous similar message [ 2995.710506] Lustre: Skipped 1 previous similar message [ 2998.705549] Lustre: server umount lustre-OST0000 complete [ 3008.462108] Lustre: server umount lustre-OST0001 complete [ 3018.877210] Lustre: server umount lustre-MDT0000 complete [ 3021.808832] LustreError: 95368:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3021.836867] LustreError: 95368:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 45 previous similar messages [ 3022.855718] LustreError: 95350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790774815 with bad export cookie 18283713087051389125 [ 3022.860406] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3022.871986] LustreError: 95350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3023.538936] Lustre: server umount lustre-MDT0001 complete [ 3032.145551] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:27:03 (1790774823) [ 3039.739410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3039.750890] Lustre: *** cfs_fail_loc=197, val=0*** [ 3055.393177] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3060.513521] LustreError: 103561:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.138@tcp: failed processing log, type 4: rc = -110 [ 3065.670439] Lustre: *** cfs_fail_loc=197, val=0*** [ 3065.677126] Lustre: Skipped 65 previous similar messages [ 3086.239352] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3086.249481] Lustre: Skipped 2 previous similar messages [ 3092.177477] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3101.714791] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3104.314810] Lustre: server umount lustre-OST0000 complete [ 3114.793684] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:28:25 (1790774905) [ 3136.360596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3141.877923] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3151.783965] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3156.503719] LustreError: 105212:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3156.520222] LustreError: 105212:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3156.533821] LustreError: 105212:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3156.862639] Lustre: server umount lustre-MDT0000 complete [ 3192.693556] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 3206.549195] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3247.185476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3247.416856] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3247.442163] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3247.517320] Lustre: lustre-MDT0000: new disk, initializing [ 3247.638879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3252.419825] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3264.528749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3264.640193] Lustre: 109387:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3264.825617] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3270.141637] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3280.395313] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3286.584809] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3295.158683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3295.465507] Lustre: 111357:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3301.348374] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3303.674279] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3311.879807] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3316.468382] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3316.614418] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3322.582688] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3322.826198] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3329.506654] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3329.520976] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3329.521745] LustreError: Skipped 4 previous similar messages [ 3329.547565] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3329.550200] Lustre: Skipped 6 previous similar messages [ 3329.575765] Lustre: Skipped 12 previous similar messages [ 3339.236921] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3339.546152] Lustre: server umount lustre-OST0000 complete [ 3350.439940] Lustre: server umount lustre-MDT0000 complete [ 3355.111819] LustreError: 110501:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3355.142524] LustreError: 110501:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 3355.157568] LustreError: 109381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790775147 with bad export cookie 18283713087051399674 [ 3355.177545] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3355.536909] Lustre: server umount lustre-MDT0001 complete [ 3365.534251] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:32:36 (1790775156) [ 3376.009559] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 3379.023089] Key type lgssc unregistered [ 3379.389730] LNet: 113845:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3379.400542] LNetError: 113845:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3379.427902] LNet: Removed LNI 192.168.202.138@tcp [ 3380.174623] Key type .llcrypt unregistered [ 3380.179075] Key type ._llcrypt unregistered [ 3404.379245] Key type ._llcrypt registered [ 3404.384160] Key type .llcrypt registered [ 3404.568985] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 3425.538430] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3427.222482] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3427.266759] alg: No test for adler32 (adler32-zlib) [ 3428.378547] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 3428.710959] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 3430.399224] Key type lgssc registered [ 3431.613351] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3475.272157] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3475.335703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3476.631285] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3476.671702] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3476.794247] Lustre: lustre-MDT0000: new disk, initializing [ 3476.972411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3476.984957] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3482.713542] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3494.459203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3494.890987] Lustre: lustre-OST0000: new disk, initializing [ 3494.909819] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3494.914579] Lustre: Skipped 1 previous similar message [ 3494.920338] Lustre: 118039:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3494.992477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3501.522594] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3502.152308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3502.158763] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3502.191303] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3512.065882] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3512.231440] Lustre: lustre-OST0001: new disk, initializing [ 3512.234334] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3512.239587] Lustre: 118902:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3512.302308] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3518.400872] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3520.630064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3520.640248] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3520.699696] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3530.722858] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3530.737963] Lustre: Skipped 1 previous similar message [ 3530.740330] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3530.753266] Lustre: Skipped 1 previous similar message [ 3532.908670] Lustre: server umount lustre-MDT0000 complete [ 3539.353781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3545.202956] LustreError: 118038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790775337 with bad export cookie 12306745257989520772 [ 3545.208927] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3545.220376] LustreError: 118038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3545.355146] Lustre: server umount lustre-OST0000 complete [ 3550.346925] Lustre: server umount lustre-OST0001 complete [ 3559.679828] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 09:35:51 (1790775351) [ 3584.246404] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 09:36:15 (1790775375) [ 3595.777962] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3616.381850] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:36:47 (1790775407) [ 3654.538975] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 3666.578452] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3718.257329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3718.486176] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3718.507761] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3718.571235] Lustre: lustre-MDT0000: new disk, initializing [ 3718.624206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3718.635313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3724.141864] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3734.958248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3735.098377] Lustre: 126660:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3735.158158] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3735.165553] Lustre: Skipped 1 previous similar message [ 3735.272606] Lustre: lustre-MDT0001: new disk, initializing [ 3735.361024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3735.404788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3735.424877] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3739.757828] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3748.731571] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3753.411936] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3762.511958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3762.805885] Lustre: lustre-OST0000: new disk, initializing [ 3762.810950] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3762.818922] Lustre: 128628:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3762.929760] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3769.972744] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3769.997925] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3770.160764] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3771.208559] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3781.553470] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3787.964481] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3788.263019] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3794.521170] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3794.709728] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3801.057570] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3801.088976] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3801.089629] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3801.119330] Lustre: Skipped 1 previous similar message [ 3801.181866] Lustre: Skipped 1 previous similar message [ 3806.197517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3806.213037] Lustre: Skipped 1 previous similar message [ 3811.297996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3811.299529] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [ 3811.305713] Lustre: Skipped 1 previous similar message [ 3811.552095] Lustre: server umount lustre-OST0000 complete [ 3816.421467] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3816.429027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3816.448401] Lustre: Skipped 1 previous similar message [ 3816.460248] Lustre: Skipped 1 previous similar message [ 3822.260169] Lustre: server umount lustre-MDT0000 complete [ 3826.643419] LustreError: 127455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790775618 with bad export cookie 12306745257989521997 [ 3826.650588] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3826.652360] LustreError: 127455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3827.346864] Lustre: server umount lustre-MDT0001 complete [ 3838.426677] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:40:28 (1790775628) [ 3845.474586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3845.893690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3851.160405] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3851.232807] LustreError: 131045:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3851.250401] LustreError: 131045:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3856.354617] LustreError: 131046:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3860.184558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3865.078764] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3873.664644] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3879.880189] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3886.730528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3887.123400] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3887.138577] Lustre: Skipped 1 previous similar message [ 3893.277658] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3900.126849] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3907.608307] Lustre: *** cfs_fail_loc=159, val=0*** [ 3924.285648] Lustre: lustre-MDT0000: Client bdf863b0-13e0-4aa0-90cf-8d67f18bf89f (at 192.168.202.38@tcp) reconnecting [ 3926.077899] Lustre: *** cfs_fail_loc=159, val=0*** [ 3926.079545] Lustre: Skipped 5 previous similar messages [ 3941.668683] Lustre: lustre-MDT0000: Client bdf863b0-13e0-4aa0-90cf-8d67f18bf89f (at 192.168.202.38@tcp) reconnecting [ 3946.465345] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3946.483110] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3946.483367] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3946.503460] Lustre: Skipped 3 previous similar messages [ 3951.343393] Lustre: server umount lustre-OST0000 complete [ 3956.710967] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3956.727803] Lustre: Skipped 2 previous similar messages [ 3961.263453] Lustre: server umount lustre-MDT0000 complete [ 3961.825838] LustreError: 132277:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3961.857683] LustreError: 132277:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3965.397945] LustreError: 131027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790775757 with bad export cookie 12306745257989522879 [ 3965.410234] LustreError: 131027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3965.413747] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3965.844947] Lustre: server umount lustre-MDT0001 complete [ 3977.189654] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 3979.998100] Key type lgssc unregistered [ 3980.354218] LNet: 134815:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3980.369696] LNetError: 134815:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3980.405562] LNet: Removed LNI 192.168.202.138@tcp [ 3981.723194] Key type .llcrypt unregistered [ 3981.724988] Key type ._llcrypt unregistered [ 4008.029681] Key type ._llcrypt registered [ 4008.032246] Key type .llcrypt registered [ 4008.218993] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4009.553784] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4009.638651] alg: No test for adler32 (adler32-zlib) [ 4010.633702] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 4010.870295] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4012.511283] Key type lgssc registered [ 4013.802275] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4021.072333] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4021.106172] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4022.540749] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4027.386559] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4031.973440] LustreError: 136243:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4031.995475] LustreError: 136243:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4035.472670] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4035.815317] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4040.318242] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4046.283956] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4052.203891] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4059.078685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4059.432456] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4065.327404] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4067.642964] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4071.934378] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4081.153787] Lustre: *** cfs_fail_loc=159, val=0*** [ 4096.807595] Lustre: lustre-MDT0000: Client ee5d9fdf-404c-4cec-acbe-ea8b444cd44d (at 192.168.202.38@tcp) reconnecting [ 4099.318266] Lustre: *** cfs_fail_loc=159, val=0*** [ 4099.322781] Lustre: Skipped 2 previous similar messages [ 4115.813050] Lustre: lustre-MDT0001: Client ee5d9fdf-404c-4cec-acbe-ea8b444cd44d (at 192.168.202.38@tcp) reconnecting [ 4117.402361] Lustre: *** cfs_fail_loc=159, val=0*** [ 4117.416719] Lustre: Skipped 1 previous similar message [ 4133.701673] Lustre: lustre-MDT0001: Client ee5d9fdf-404c-4cec-acbe-ea8b444cd44d (at 192.168.202.38@tcp) reconnecting [ 4139.300379] Lustre: 139282:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4142.248519] Lustre: *** cfs_fail_loc=159, val=0*** [ 4142.253075] Lustre: Skipped 4 previous similar messages [ 4157.745379] Lustre: lustre-MDT0000: Client c662f9cf-d4e0-416e-bd8c-6a52ebb0db27 (at 192.168.202.38@tcp) reconnecting [ 4159.623893] Lustre: *** cfs_fail_loc=159, val=0*** [ 4159.627562] Lustre: Skipped 13 previous similar messages [ 4176.198846] Lustre: lustre-MDT0000: Client c662f9cf-d4e0-416e-bd8c-6a52ebb0db27 (at 192.168.202.38@tcp) reconnecting [ 4177.923482] Lustre: 139575:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4183.523506] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4183.529178] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4183.535642] LustreError: Skipped 1 previous similar message [ 4183.561647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4185.570750] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4185.573084] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4185.574448] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4185.614640] LustreError: Skipped 2 previous similar messages [ 4188.710126] Lustre: server umount lustre-OST0000 complete [ 4194.784612] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4194.792725] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4194.808542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4194.819595] Lustre: Skipped 1 previous similar message [ 4198.702919] Lustre: server umount lustre-MDT0000 complete [ 4200.936297] LustreError: 136243:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4200.975309] LustreError: 136243:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4203.361391] LustreError: 136225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790775995 with bad export cookie 3026206072105711579 [ 4203.376491] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4203.387247] LustreError: 136225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4203.713465] Lustre: server umount lustre-MDT0001 complete [ 4219.073863] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4222.829704] Key type lgssc unregistered [ 4223.177934] LNet: 140628:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4223.184332] LNetError: 140628:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4223.195789] LNet: Removed LNI 192.168.202.138@tcp [ 4224.062189] Key type .llcrypt unregistered [ 4224.064786] Key type ._llcrypt unregistered [ 4251.145394] Key type ._llcrypt registered [ 4251.147555] Key type .llcrypt registered [ 4251.290596] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4252.782461] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4252.802047] alg: No test for adler32 (adler32-zlib) [ 4253.886258] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 4254.083573] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4255.737475] Key type lgssc registered [ 4257.030798] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4265.049295] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4265.070407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4266.460068] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4271.050921] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4276.705404] LustreError: 142061:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4276.745226] LustreError: 142061:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4281.603255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4281.829479] LustreError: 142060:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4282.183828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4288.718363] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4296.429171] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4302.766731] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4310.211140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4310.472821] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4311.557211] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4316.400405] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4323.881226] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4326.894795] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4334.420182] Lustre: 144644:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4337.816372] Lustre: 144692:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4339.128583] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4339.939211] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4343.233760] Lustre: 144838:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4343.952552] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4344.713319] Lustre: 144936:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4352.487578] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4352.491248] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4352.494178] LustreError: Skipped 1 previous similar message [ 4352.495965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4352.516741] Lustre: Skipped 1 previous similar message [ 4354.127822] Lustre: server umount lustre-OST0000 complete [ 4359.137838] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4359.155285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4359.164340] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4359.167570] Lustre: Skipped 1 previous similar message [ 4362.721420] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4362.723493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4362.735390] Lustre: Skipped 1 previous similar message [ 4364.238202] Lustre: server umount lustre-MDT0000 complete [ 4367.603250] LustreError: 142042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790776159 with bad export cookie 10274560982058843083 [ 4367.613422] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4367.618225] LustreError: 142042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4368.004324] Lustre: server umount lustre-MDT0001 complete [ 4379.139637] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4382.476193] Key type lgssc unregistered [ 4382.782055] LNet: 145998:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4382.788246] LNetError: 145998:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4383.847549] LNet: Removed LNI 192.168.202.138@tcp [ 4384.822753] Key type .llcrypt unregistered [ 4384.828603] Key type ._llcrypt unregistered [ 4409.090852] Key type ._llcrypt registered [ 4409.093539] Key type .llcrypt registered [ 4409.169871] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4410.583316] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4410.726380] alg: No test for adler32 (adler32-zlib) [ 4411.826936] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 4412.137568] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4413.871269] Key type lgssc registered [ 4415.242689] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4424.063857] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4424.077453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4425.579579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4430.497950] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4434.916255] LustreError: 147430:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4434.953661] LustreError: 147430:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4439.706306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4440.161589] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4444.292498] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4452.739461] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4459.577259] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4468.280086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4468.794434] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4469.809591] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4474.468707] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4481.807881] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4485.165982] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4490.344765] Lustre: *** cfs_fail_loc=159, val=0*** [ 4490.361234] Lustre: Skipped 2 previous similar messages [ 4506.924706] Lustre: lustre-MDT0000: Client ea9223b8-e336-4bd9-b50a-62def2626eaf (at 192.168.202.38@tcp) reconnecting [ 4526.054060] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4526.063861] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4526.068103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4526.081564] Lustre: Skipped 1 previous similar message [ 4530.179982] Lustre: server umount lustre-OST0000 complete [ 4536.289564] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4536.306910] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4536.314885] Lustre: Skipped 2 previous similar messages [ 4541.408884] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4541.413204] Lustre: Skipped 1 previous similar message [ 4546.529056] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4546.544924] Lustre: Skipped 1 previous similar message [ 4549.088461] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4549.325205] Lustre: server umount lustre-MDT0000 complete [ 4551.651553] LustreError: 147430:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4551.694551] LustreError: 147430:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4553.742796] LustreError: 147409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790776346 with bad export cookie 16757575998963488833 [ 4553.749891] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4554.250125] Lustre: server umount lustre-MDT0001 complete [ 4565.271875] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4568.353908] Key type lgssc unregistered [ 4568.653860] LNet: 151098:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4568.659071] LNetError: 151098:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4568.671180] LNet: Removed LNI 192.168.202.138@tcp [ 4569.453326] Key type .llcrypt unregistered [ 4569.456411] Key type ._llcrypt unregistered [ 4595.372478] Key type ._llcrypt registered [ 4595.374935] Key type .llcrypt registered [ 4595.470826] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4596.516260] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4596.541991] alg: No test for adler32 (adler32-zlib) [ 4597.857147] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 4598.192198] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4599.904231] Key type lgssc registered [ 4601.087800] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4610.053091] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4610.073460] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4611.704040] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4617.709362] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4621.284566] LustreError: 152527:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4621.314179] LustreError: 152527:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4626.408572] LustreError: 152528:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4628.091032] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4628.437788] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4633.647318] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4641.121129] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4646.382330] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4653.131455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4653.422717] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4656.505280] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4659.751255] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4666.730641] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4672.900979] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4690.705779] Lustre: lustre-OST0000: evicting b63664e0-8840-4052-bdf8-ebc1770a0fff (at 192.168.202.38@tcp) by administrative request [ 4690.715182] Lustre: Skipped 1 previous similar message [ 4723.169072] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4723.188142] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4723.215443] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.193410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.209056] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4724.219250] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4726.955457] Lustre: server umount lustre-OST0000 complete [ 4734.432693] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4734.437990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4734.461793] Lustre: Skipped 1 previous similar message [ 4734.478805] Lustre: Skipped 1 previous similar message [ 4739.571352] LustreError: 153925:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4739.589876] LustreError: 153925:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4739.930905] Lustre: server umount lustre-MDT0000 complete [ 4743.688904] LustreError: 152508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790776535 with bad export cookie 12781410797944731323 [ 4743.696479] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4743.698449] LustreError: 152508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4744.055737] Lustre: server umount lustre-MDT0001 complete [ 4756.356766] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4758.977388] Key type lgssc unregistered [ 4759.340436] LNet: 156598:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4759.345596] LNetError: 156598:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4759.360395] LNet: Removed LNI 192.168.202.138@tcp [ 4760.209406] Key type .llcrypt unregistered [ 4760.211414] Key type ._llcrypt unregistered [ 4828.473649] Key type ._llcrypt registered [ 4828.475122] Key type .llcrypt registered [ 4828.547144] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4829.904759] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4830.169573] alg: No test for adler32 (adler32-zlib) [ 4831.404402] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 4831.773424] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4833.575309] Key type lgssc registered [ 4834.665151] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4842.743977] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4842.791827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4844.236676] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4849.360270] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4854.754758] LustreError: 159103:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4854.784953] LustreError: 159103:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4859.781837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4859.882399] LustreError: 159104:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4860.350653] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4864.807661] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4871.377583] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4878.514161] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4888.274651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4888.838218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4889.841856] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4896.853751] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4905.204643] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4906.071842] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4920.802284] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4920.803381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4920.815871] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4920.830665] Lustre: Skipped 1 previous similar message [ 4923.984739] Lustre: server umount lustre-OST0000 complete [ 4931.042052] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4931.057878] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4931.064065] Lustre: Skipped 1 previous similar message [ 4931.073060] Lustre: Skipped 1 previous similar message [ 4935.414359] Lustre: server umount lustre-MDT0000 complete [ 4936.165616] LustreError: 159104:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4936.187361] LustreError: 159104:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4938.983414] LustreError: 159085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790776731 with bad export cookie 16306732139005611684 [ 4938.995107] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4939.307083] Lustre: server umount lustre-MDT0001 complete [ 4950.310668] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4952.752811] Key type lgssc unregistered [ 4953.052800] LNet: 162646:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4953.061382] LNetError: 162646:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4953.081535] LNet: Removed LNI 192.168.202.138@tcp [ 4953.915772] Key type .llcrypt unregistered [ 4953.918689] Key type ._llcrypt unregistered [ 4983.255355] Key type ._llcrypt registered [ 4983.257478] Key type .llcrypt registered [ 4983.393183] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4984.687276] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4984.805815] alg: No test for adler32 (adler32-zlib) [ 4986.203921] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 4986.484574] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4988.200197] Key type lgssc registered [ 4989.451080] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5005.589430] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 09:59:56 (1790776796) [ 5024.248856] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 5040.961621] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5080.254261] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5080.277470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5081.604503] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5081.653504] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5081.733786] Lustre: lustre-MDT0000: new disk, initializing [ 5081.780933] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5081.798281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5087.457838] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5099.491779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5099.756398] Lustre: 166958:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5099.892517] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5099.899483] Lustre: Skipped 1 previous similar message [ 5100.088603] Lustre: lustre-MDT0001: new disk, initializing [ 5100.342487] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5100.421568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5100.455932] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5105.834806] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5115.980978] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5121.435378] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5128.745441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5129.206402] Lustre: lustre-OST0000: new disk, initializing [ 5129.209616] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5129.216568] Lustre: 168930:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5129.374579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5134.962186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5134.977021] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5135.119045] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5136.314348] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5147.079407] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5160.936347] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5160.940994] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5160.966866] LustreError: Skipped 1 previous similar message [ 5161.030839] Lustre: Skipped 1 previous similar message [ 5161.041255] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5161.056027] Lustre: Skipped 1 previous similar message [ 5164.375887] Lustre: server umount lustre-OST0000 complete [ 5171.170552] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5171.175935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5171.193539] Lustre: Skipped 1 previous similar message [ 5171.222369] Lustre: Skipped 1 previous similar message [ 5174.779860] Lustre: server umount lustre-MDT0000 complete [ 5178.618397] LustreError: 168929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790776970 with bad export cookie 10471612160626684553 [ 5178.627428] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5178.631846] LustreError: 168929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5178.985552] Lustre: server umount lustre-MDT0001 complete [ 5191.049394] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 5194.240129] Key type lgssc unregistered [ 5194.730238] LNet: 170833:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5194.741225] LNetError: 170833:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5194.762742] LNet: Removed LNI 192.168.202.138@tcp [ 5195.885206] Key type .llcrypt unregistered [ 5195.887673] Key type ._llcrypt unregistered [ 5222.759707] Key type ._llcrypt registered [ 5222.767086] Key type .llcrypt registered [ 5222.913302] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5224.278777] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5224.444574] alg: No test for adler32 (adler32-zlib) [ 5225.527988] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 5225.829376] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 5227.593267] Key type lgssc registered [ 5229.371130] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5237.669614] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5237.691972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5239.260927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5244.906901] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5249.512159] LustreError: 172264:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5249.528884] LustreError: 172264:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5250.531956] LustreError: 172263:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5254.050323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5254.482525] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5259.301582] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5267.886078] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5275.052632] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5282.639182] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5282.938711] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5285.006833] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5290.399209] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5298.803432] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5305.125995] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5305.144595] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5305.152507] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5305.163969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.178202] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5306.510454] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5306.590553] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5306.628052] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5306.646079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.660676] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5308.038446] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5308.064512] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5308.079980] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5308.085202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.093882] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5308.952215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.169438] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5310.177664] Lustre: Skipped 1 previous similar message [ 5310.192633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5310.210524] Lustre: Skipped 1 previous similar message [ 5310.214933] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5310.220800] Lustre: Skipped 1 previous similar message [ 5310.225254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.236771] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5310.243047] Lustre: Skipped 1 previous similar message [ 5311.360509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.425134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.576715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.055261] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5315.062840] Lustre: Skipped 3 previous similar messages [ 5315.076833] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5315.083502] Lustre: Skipped 3 previous similar messages [ 5315.086671] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5315.090180] Lustre: Skipped 3 previous similar messages [ 5315.092837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.098463] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5315.100728] Lustre: Skipped 3 previous similar messages [ 5315.883937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.363535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.336839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.194751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.332195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.271158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.382793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.329344] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5323.332519] Lustre: Skipped 7 previous similar messages [ 5323.354928] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5323.363690] Lustre: Skipped 7 previous similar messages [ 5323.367214] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5323.371971] Lustre: Skipped 7 previous similar messages [ 5323.374785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.386101] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5323.394087] Lustre: Skipped 7 previous similar messages [ 5324.454219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.594738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.686132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.882957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.105995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.481160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.859723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.653529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.443435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.292785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.524737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.613888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.692207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.909432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.065584] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5340.077110] Lustre: Skipped 14 previous similar messages [ 5340.106864] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5340.133194] Lustre: Skipped 14 previous similar messages [ 5340.136859] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5340.142734] Lustre: Skipped 14 previous similar messages [ 5340.146705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.160858] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5340.166061] Lustre: Skipped 14 previous similar messages [ 5341.487303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.657624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.139045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.228652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.308524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.319040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.173855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.049719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.855553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.775342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.736239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.481150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.352925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.181254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.991721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.016031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.801161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.652163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.414456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.284233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.052705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.936309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.762323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.561616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.334091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.244865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.175879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.863096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.775922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.621068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.393772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.217148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.902410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.902748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.742661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.594407] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5372.600608] Lustre: Skipped 35 previous similar messages [ 5372.619142] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5372.631989] Lustre: Skipped 35 previous similar messages [ 5372.636815] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5372.640553] Lustre: Skipped 35 previous similar messages [ 5372.645638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.658329] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5372.669800] Lustre: Skipped 35 previous similar messages [ 5373.595936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.601510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.356459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.381251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.252064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.175267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.055709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.989652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.867970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.606273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.479347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.461668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.438184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.441500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.318437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.373616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.508601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.309288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.363257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.323248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.127626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.097675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.929889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.965307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.686341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.572705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.593204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.625864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.778953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.577817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.541866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.450224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.414389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.308488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.420209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.297800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.242551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.133323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.198474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.918244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.871370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.833242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.825265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.794914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.825597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.811273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.792221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.729271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.704293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.856174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.951958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.011894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.031933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.014699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.194513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.122698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.202940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.215237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.258175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.344672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.223686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.147461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.152762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.129365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.078865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.914616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.861363] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5436.872353] Lustre: Skipped 66 previous similar messages [ 5436.890836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5436.902311] Lustre: Skipped 66 previous similar messages [ 5436.906823] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5436.909932] Lustre: Skipped 66 previous similar messages [ 5436.914095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.921042] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5436.923941] Lustre: Skipped 66 previous similar messages [ 5437.700674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.813671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.771788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.800037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.734588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.596704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.580628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.614995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.812503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.005082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.242842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.448334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.617458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.809301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.725264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.690255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.630900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.562464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.352597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.164937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.979485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.015667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.881129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.903729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.862127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.968518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.962652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.919047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.761785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.654085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.428231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.485488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.368980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.401601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.347787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.365334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.362296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.404996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.295956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.492974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.800366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.890681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.120090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.265433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.661347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.127325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.358251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.740598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.025611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.934503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.987397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.607177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.636973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.521286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.418357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.567292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.575926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.540736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.417553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.518603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.215399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.322984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.683589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.832671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.022576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.913497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.730609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.535295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.724658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.122421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.698059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.775375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.601188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.664112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.474836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.518636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.675535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.874300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.856832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.834192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.217581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.074438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.761480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.645336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.650653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.463619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.377441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.438420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.317908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.253207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.110091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.097433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.130690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.852600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.734858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.775404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.812839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.930581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.922978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.639644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.704350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.531299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.355142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.230565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.250694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.211258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.137348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.164231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.277632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.125607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.139491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.946206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.793904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.641074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.508487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.309061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.138543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.086751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.037965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.003701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.261680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.267895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.381757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.590211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.802493] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5565.820154] Lustre: Skipped 124 previous similar messages [ 5565.835845] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5565.848567] Lustre: Skipped 124 previous similar messages [ 5565.853980] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5565.858653] Lustre: Skipped 124 previous similar messages [ 5565.862810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.871589] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5565.879130] Lustre: Skipped 124 previous similar messages [ 5566.775190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.864843] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5567.882763] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5567.887902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.775714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.784655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.824679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.601865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.420131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.247808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.542830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.465488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.773520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.963722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.110542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.031252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.024505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.040890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.063691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.085087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.953430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.764701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.721777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.738988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.749633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.554867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.474351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.393855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.238406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.200328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.743131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.895461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.901474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.934948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.011849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.248334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.310485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.301226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.321474] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5603.334731] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5603.358223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.554202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.882742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.110766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.181562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.105834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.416871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.208786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.250924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.091728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.060857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.191776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.905238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.569619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.592713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.799885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.279886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.520079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.662580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.402031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.319145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.174906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.006973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.840388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.890929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.856141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.763935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.855171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.780727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.636390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.386887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.353081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.218887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.108247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.900449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.957389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.864201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.858251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.860925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.931834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.649611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.644842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.475371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.542417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.407975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.251836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.150250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.140352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.060079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.846266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.707659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.567635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.456996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.369075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.581484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.314437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.384171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.258331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.462420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.480247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.491485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.429906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.491822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.514329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.377181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.505634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.314502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.349398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.226675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.212850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.335903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.387529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.698380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.021236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.251867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.483898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.818413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.888148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.142811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.382672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.481326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.570835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.662986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.780758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.669564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.626280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.408569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.302952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.193962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.376718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.364470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.315067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.078645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.085724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.316631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.209429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.250555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.231912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.341492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.306843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.253438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.058291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.992576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.906773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.931662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.795694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.862823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.894569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.022912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.983105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.209343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.099139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.068963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.865626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.102659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.859474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.810058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.697689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.916722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.044533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.207635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.789730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.276468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.512651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.551257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.785965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.818950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.201569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.108718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.068903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.471948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.282218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.281233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.256818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.206503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.911855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.837124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.875410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.951136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.187438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.093183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.675550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.173402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.047515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.803245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.887525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.498934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.396187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.304953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.228556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.016230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.849253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.715293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.697247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.584020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.338403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.022963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.818677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.718242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.626942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.527229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.230272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.215467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.006220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.844143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.751629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.718648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.655368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.582360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.601358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.567693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.372682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.646098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.594675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.582679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.427507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.490374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.358613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.412490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.242027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.188557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.930789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.852544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.586626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.565945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.272515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.103267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.877079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.652812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.459445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.238948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.161564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.113044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.325427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.636165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.526766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.765931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.933505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.766135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.791173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.693054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.783601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.850226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.861080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.132255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.963923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.967290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.879277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.983314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.071784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.077851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.040648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.865579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.690729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.542929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.523962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.557772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.509932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.349127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.256823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.221186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.313778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.206701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.237887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.168932] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5822.173061] Lustre: Skipped 260 previous similar messages [ 5822.191682] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5822.206266] Lustre: Skipped 260 previous similar messages [ 5822.209751] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5822.217694] Lustre: Skipped 260 previous similar messages [ 5822.222613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.229796] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5822.240269] Lustre: Skipped 260 previous similar messages [ 5823.272414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.238356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.149420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.156780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.035326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.013461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.835227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.568410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.407218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.239576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.194662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.062629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.102655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.061666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.086786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.258433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.115977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.336187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.271740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.370418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.224219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.141044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.183212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.096634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.986173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.979732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.905962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.026448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.851297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.015104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.789290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.552380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.324739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.272369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.177482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.118427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.964849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.981498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.961709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.026716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.975729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.138652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.275048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.286646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.213537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.187833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.250304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.151289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.162362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.007577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.106197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.078491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.064810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.039296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.867568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.456718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.214782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.065996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.097177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.929204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.020550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.997335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.011173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.125355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.045233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.181738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.219572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.327129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.327407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.186843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.983765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.850178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.737578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.736367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.622559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.347616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.140989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.080210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.090693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.170071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.174280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.003567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.880152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.874269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.693454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.636933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.513450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.559773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.398606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.309579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.051957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.068320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.837088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.833360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.021055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.292538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.211851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.141942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.989240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.042134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.023218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.956770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.905543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.674758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.655025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.488838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.449838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.436770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.386223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.500641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.346228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.205298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.137283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.287136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.197569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.475851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.420379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.420709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.258492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.149615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.965430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.967302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.925308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.183857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.241090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.572910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.970371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.314878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.975214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.191271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.235785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.500686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.746680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.936848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.134679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.862961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.615115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.550658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.501473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.344337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.192502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.964087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.897517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.564749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.428674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.257076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.270353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.064651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.857483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.020937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.151123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.182287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.190224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.228316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.985173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.895627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.842361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.996928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.072215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.324795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.348580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.709611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.433221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.192232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.084446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.098946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.932619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.760888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.777929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.587833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.396198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.246745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.057293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.086967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.024806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.989988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.952597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.919950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.050677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.129405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.043777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.340129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.313407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.564313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.505715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.225083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.155266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.127333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.798811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.571844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.572039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.379188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.213707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.132679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.896378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.910328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.818379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.835733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.671935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.536976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.260423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.078423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.829984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.812414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.703566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.614879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.372433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.336511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.108886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.062327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.869748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.909663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.789976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.698319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.667545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.604213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.399915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.291876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.160678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.154553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.895751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.735363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.562199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.318445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.135320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.976201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.921390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.765329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.631677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.524947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.641299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.510560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.421400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.073818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.983288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.840157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.632079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.447334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.244598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.101314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.039496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.767508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.653355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.337059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.267218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.979383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.952955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.974782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.032935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.028892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.139772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.098262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.009315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.026853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.928341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.659952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.458099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.318087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.394849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.440691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.539560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.528236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.391385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.187470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.956889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.797642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.790623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.623278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.359147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.469789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.400690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.396789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.262480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.303198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.156561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.095823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.079942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.273130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.097194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.168809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.108306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.167654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.317671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.175679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.039710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.842215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.755258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.786877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.049632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.953784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.081057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.154903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.213645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.108779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.074181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.076608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.054916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.125832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.167765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.192653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.186415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.556842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.804917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.917145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.874629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.030292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.959271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.969910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.885941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.937573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.097851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.980888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.977373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.769646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.766856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.672624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.787376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.682359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.673561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.536498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.427331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.251134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.235357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.620730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.848117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.853949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.732272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.519398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.466686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.397574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.478778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.381735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.381132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.215642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.134233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.903480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.764514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.582380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.471170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.357974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.156403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.919787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.863576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.790582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.701936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.674531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.443510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.187699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.955709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.761935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.495489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.251201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.263264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.223683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.253994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.039350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.979436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.888937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.805798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.547679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.322219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.219853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.208786] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6165.221108] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6165.226721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.020705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.787239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.415458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.418782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.168158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.015053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.868341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.881107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.825115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.728453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.597923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.655915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.541965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.454779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.297177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.189981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.089614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.145244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.048984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.253622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.268974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.315550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.420470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.636666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.872823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.951420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.098152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.381793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.257083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.334476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.121883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.017988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.870991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.711182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.725947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.520519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.506285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.395065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.362680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.268816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.236310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.992503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.902345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.004088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.137633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.266538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.122504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.136673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.125993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.045116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.001916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.857195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.543726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.437796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.182813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.126155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.090315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.066649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.880888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.851716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.789140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.880487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.662119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.592295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.594360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.518256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.341809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.535778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.589359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.535906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.520816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.436597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.340106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.312323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.132336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.061167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.796381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.607152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.488325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.279470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.185770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.370119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.241646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.093865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.254175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.155948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.170879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.153871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.976936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.798622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.588245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.431649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.384756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.950387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.863474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.706916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.525892] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6255.532441] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6255.533428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.349054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.281489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.235927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.151046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.921016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.699810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.304329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.163142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.883755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.638528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.326337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.114518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.864700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.808171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.053126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.257967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.395550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.356372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.570879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.121630] LustreError: 174085:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6591.134117] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7018.982034] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7018.989954] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7019.003306] Lustre: Skipped 480 previous similar messages [ 7019.024505] Lustre: Skipped 1 previous similar message [ 7022.098761] Lustre: server umount lustre-OST0000 complete [ 7026.144635] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7026.151775] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7029.220397] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7032.557893] Lustre: server umount lustre-MDT0000 complete [ 7034.336053] LustreError: 172263:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7034.358076] LustreError: 172263:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7036.868056] LustreError: 172244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790778829 with bad export cookie 6628546445801879308 [ 7036.869627] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7036.881065] LustreError: 172244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7037.557387] Lustre: server umount lustre-MDT0001 complete [ 7050.275888] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 7053.133379] Key type lgssc unregistered [ 7053.365448] LNet: 227480:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7053.371325] LNetError: 227480:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7053.385606] LNet: Removed LNI 192.168.202.138@tcp [ 7054.296876] Key type .llcrypt unregistered [ 7054.300463] Key type ._llcrypt unregistered [ 7079.825883] Key type ._llcrypt registered [ 7079.827740] Key type .llcrypt registered [ 7079.950684] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7081.373897] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7081.406017] alg: No test for adler32 (adler32-zlib) [ 7082.431377] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 7082.665607] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 7084.329797] Key type lgssc registered [ 7085.770961] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7093.775950] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7093.790965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7095.284900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7099.724892] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7105.505320] LustreError: 228909:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7105.522489] LustreError: 228909:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7107.525023] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7107.860393] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7112.180718] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7119.483751] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7125.445695] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7133.808121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7134.348877] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7140.612219] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7144.087195] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7154.494435] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7159.100787] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7160.957300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7160.969922] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7160.976060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7160.996883] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7160.998633] LustreError: 228929:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7163.364257] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7163.371940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7163.383646] Lustre: Skipped 1 previous similar message [ 7163.399874] Lustre: Skipped 1 previous similar message [ 7166.740165] Lustre: lustre-OST0000: Not available for connect from 192.168.202.38@tcp (stopping) [ 7168.495327] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7171.836489] Lustre: lustre-OST0000: Not available for connect from 192.168.202.38@tcp (stopping) [ 7171.846985] Lustre: Skipped 1 previous similar message [ 7176.944985] Lustre: lustre-OST0000: Not available for connect from 192.168.202.38@tcp (stopping) [ 7176.947574] Lustre: Skipped 2 previous similar messages [ 7177.183373] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7177.400419] Lustre: server umount lustre-OST0000 complete [ 7181.080211] LustreError: 228929:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7212.540580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7212.823168] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7219.940951] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7228.115768] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7228.216557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.236467] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.241292] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7228.267490] Lustre: Skipped 1 previous similar message [ 7234.375721] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 7239.845720] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c9e1108b000.ost_server_uuid 50 [ 7241.627298] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c9e1108b000.ost_server_uuid in FULL state after 0 sec [ 7246.816962] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7246.824929] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7246.834702] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7248.874596] LustreError: 232120:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7248.881043] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7248.882119] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7249.070708] Lustre: server umount lustre-OST0000 complete [ 7253.993093] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7254.018440] Lustre: Skipped 1 previous similar message [ 7264.226801] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7264.249585] Lustre: Skipped 5 previous similar messages [ 7267.295578] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7267.794216] Lustre: server umount lustre-MDT0000 complete [ 7269.345589] LustreError: 228910:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7269.378568] LustreError: 228910:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7272.372465] LustreError: 228889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779064 with bad export cookie 14906391192283194442 [ 7272.382044] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7272.384876] LustreError: 228889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7272.680938] Lustre: server umount lustre-MDT0001 complete [ 7286.530189] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 7289.219416] Key type lgssc unregistered [ 7289.494580] LNet: 234122:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7289.507735] LNetError: 234122:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7289.532620] LNet: Removed LNI 192.168.202.138@tcp [ 7290.623276] Key type .llcrypt unregistered [ 7290.624937] Key type ._llcrypt unregistered [ 7328.048744] Key type ._llcrypt registered [ 7328.055758] Key type .llcrypt registered [ 7328.199242] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7329.870909] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7329.924901] alg: No test for adler32 (adler32-zlib) [ 7331.009809] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 7331.410573] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 7333.336281] Key type lgssc registered [ 7335.404717] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7346.742540] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7346.761623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7348.290634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7354.388839] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7358.435861] LustreError: 235869:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7358.454234] LustreError: 235869:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7362.904518] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7363.163425] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7367.927612] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7371.182907] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7378.154188] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7378.661660] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7386.299635] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7393.035955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7394.869433] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7395.080456] Lustre: lustre-OST0001: new disk, initializing [ 7395.084688] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7395.090318] Lustre: 238175:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7395.217462] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7401.458082] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7402.625565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7402.636387] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7402.790819] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7412.398463] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7420.320833] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7432.989092] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7432.999112] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7433.010777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7433.226463] Lustre: server umount lustre-MDT0000 complete [ 7433.698421] LustreError: 235869:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7433.706353] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7433.716408] LustreError: 235869:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7433.735751] Lustre: Skipped 1 previous similar message [ 7436.731188] LustreError: 235848:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779228 with bad export cookie 7001925072914608948 [ 7436.739389] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7436.740170] LustreError: 235848:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7437.077939] Lustre: server umount lustre-MDT0001 complete [ 7451.940667] Lustre: server umount lustre-OST0000 complete [ 7455.199353] Lustre: 235120:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790779231/real 1790779231] req@ffff9e3e97609180 x1877768009772800/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790779247 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7455.232650] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7455.254065] Lustre: Skipped 1 previous similar message [ 7456.069115] Lustre: server umount lustre-OST0001 complete [ 7478.803439] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7489.311271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7489.693134] LustreError: 241286:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7489.713159] LustreError: 241286:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7489.775363] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7493.710436] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7495.139073] LustreError: 241287:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7499.239300] LustreError: 241286:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7501.184104] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7501.501979] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7506.087719] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7508.279473] Lustre: 242428:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7513.503749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7514.854331] LustreError: 242781:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7514.868147] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7519.241800] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7526.423154] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7527.691287] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7527.910059] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7527.917022] Lustre: Skipped 1 previous similar message [ 7535.668445] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7544.687716] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7549.117375] Lustre: 244296:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7563.242608] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7563.266422] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7563.292777] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7565.476536] Lustre: server umount mylustre-MDT0000 complete [ 7568.866234] LustreError: 241282:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7568.874138] LustreError: 241282:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 7569.107532] LustreError: 241261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779361 with bad export cookie 7001925072914611811 [ 7569.116682] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7569.119506] LustreError: 241261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7569.384442] Lustre: server umount mylustre-MDT0001 complete [ 7582.311766] Lustre: server umount mylustre-OST0000 complete [ 7595.002733] Lustre: server umount mylustre-OST0001 complete [ 7621.941285] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7632.944553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7633.660463] LustreError: 246608:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7633.727213] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7638.841672] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7647.311267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7651.981496] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7655.085968] Lustre: 247750:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7661.184336] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7667.509723] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7671.848600] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7676.086236] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7678.344864] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7682.731122] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7690.319197] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7694.010997] Lustre: 249622:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7704.042317] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7704.046467] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7704.057186] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7704.063336] Lustre: Skipped 5 previous similar messages [ 7704.069024] Lustre: Skipped 4 previous similar messages [ 7707.238490] Lustre: server umount tfs-MDT0000 complete [ 7709.163476] LustreError: 246605:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7709.195203] LustreError: 246605:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 7711.019250] LustreError: 246585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779503 with bad export cookie 7001925072914613764 [ 7711.020253] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7711.028112] LustreError: 246585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7711.414740] Lustre: server umount tfs-MDT0001 complete [ 7726.459555] Lustre: server umount tfs-OST0000 complete [ 7740.912770] Lustre: server umount tfs-OST0001 complete [ 7760.513128] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7770.304282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7770.823031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7770.825959] Lustre: Skipped 3 previous similar messages [ 7775.503866] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7783.427197] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7787.464412] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7789.782285] Lustre: 253016:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7795.116846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7796.462255] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7801.534514] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7808.001579] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7810.555693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7816.774849] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7823.851981] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7826.901485] Lustre: 254890:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7833.808939] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:47:04 (1790779624) [ 7852.015557] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7852.020637] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7852.037060] Lustre: Skipped 3 previous similar messages [ 7852.065119] Lustre: Skipped 5 previous similar messages [ 7857.121639] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7857.128098] Lustre: Skipped 3 previous similar messages [ 7858.203050] Lustre: server umount lustre-MDT0000 complete [ 7862.248229] LustreError: 251875:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7862.281111] LustreError: 251875:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 11 previous similar messages [ 7862.724643] LustreError: 254892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779654 with bad export cookie 7001925072914615458 [ 7862.726368] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7862.739750] LustreError: 254892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7867.391919] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7867.404353] Lustre: Skipped 1 previous similar message [ 7877.087221] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7877.281427] Lustre: server umount lustre-MDT0001 complete [ 7887.508575] Lustre: server umount lustre-OST0000 complete [ 7897.445981] Lustre: server umount lustre-OST0001 complete [ 7903.580602] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 7912.369160] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7942.908028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7943.143737] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7943.166132] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7943.235524] Lustre: lustre-MDT0000: new disk, initializing [ 7943.321259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7943.328385] Lustre: Skipped 3 previous similar messages [ 7943.341401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7947.579183] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7957.842770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7958.001166] Lustre: 258075:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7958.057640] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7958.062192] Lustre: Skipped 1 previous similar message [ 7958.184245] Lustre: lustre-MDT0001: new disk, initializing [ 7958.308671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7958.320515] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7962.827484] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7973.425924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7973.626665] Lustre: lustre-OST0000: new disk, initializing [ 7973.629804] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7973.633750] Lustre: 259657:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7975.388886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7975.408438] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7975.555700] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7980.865782] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7991.930619] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7992.169748] Lustre: lustre-OST0001: new disk, initializing [ 7992.173319] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7992.186990] Lustre: 260528:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7994.330909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7994.346068] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7994.502188] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7999.552668] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8012.200750] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:50:03 (1790779803) [ 8025.436955] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:50:16 (1790779816) [ 8040.438938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8040.445617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8040.455450] Lustre: Skipped 4 previous similar messages [ 8040.465983] Lustre: Skipped 5 previous similar messages [ 8046.553459] Lustre: server umount lustre-OST0000 complete [ 8050.684152] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8056.268849] Lustre: server umount lustre-MDT0000 complete [ 8060.474581] LustreError: 259656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779852 with bad export cookie 7001925072914617012 [ 8060.480911] LustreError: 259656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8060.481216] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8066.841815] Lustre: server umount lustre-MDT0001 complete [ 8077.325147] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 8151.152301] Lustre: server umount lustre-OST0001 complete [ 8157.832209] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 8171.461611] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 8209.169953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8209.494969] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8209.532615] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8209.665798] Lustre: lustre-MDT0000: new disk, initializing [ 8209.806399] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8209.818831] Lustre: Skipped 3 previous similar messages [ 8209.859127] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8215.671968] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8227.943829] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8228.106606] Lustre: 265381:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8228.172596] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8228.180964] Lustre: Skipped 1 previous similar message [ 8228.247491] Lustre: lustre-MDT0001: new disk, initializing [ 8228.381060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8228.397411] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8233.695630] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8244.227800] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8250.233723] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8257.539483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8257.792861] Lustre: 267349:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8258.988694] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8259.142419] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8264.043757] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8274.436722] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8284.644043] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8284.650636] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8284.654402] LustreError: Skipped 1 previous similar message [ 8284.657656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8284.657666] Lustre: Skipped 10 previous similar messages [ 8284.713165] Lustre: Skipped 5 previous similar messages [ 8289.081543] Lustre: server umount lustre-OST0000 complete [ 8294.891746] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8300.009214] LustreError: 266258:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8300.029086] LustreError: 266258:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 11 previous similar messages [ 8300.090197] Lustre: server umount lustre-MDT0000 complete [ 8304.101297] LustreError: 265371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780096 with bad export cookie 7001925072914618482 [ 8304.108240] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8304.116543] LustreError: 265371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8304.581439] Lustre: server umount lustre-MDT0001 complete [ 8314.943440] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 8317.527623] Key type lgssc unregistered [ 8317.757518] LNet: 269109:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8317.764321] LNetError: 269109:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8317.798890] LNet: Removed LNI 192.168.202.138@tcp [ 8318.713862] Key type .llcrypt unregistered [ 8318.716413] Key type ._llcrypt unregistered [ 8350.282074] Key type ._llcrypt registered [ 8350.291501] Key type .llcrypt registered [ 8350.457368] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 8352.822197] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8352.836772] alg: No test for adler32 (adler32-zlib) [ 8354.115056] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 8354.373708] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 8356.023202] Key type lgssc registered [ 8357.575669] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8367.025980] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8367.061180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8368.705942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8373.572825] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8377.834944] LustreError: 270588:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8377.847561] LustreError: 270588:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8382.956592] LustreError: 270589:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8383.080370] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8383.575618] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8389.514886] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8398.452321] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8404.656770] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8411.430604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8411.804226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8412.850883] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8419.169982] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8426.973123] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8428.057288] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8433.636084] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8433.642235] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8433.653820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8433.673586] Lustre: Skipped 1 previous similar message [ 8436.311981] Lustre: server umount lustre-OST0000 complete [ 8443.874182] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8443.897864] Lustre: Skipped 1 previous similar message [ 8443.910965] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8443.918329] Lustre: Skipped 1 previous similar message [ 8447.795480] Lustre: server umount lustre-MDT0000 complete [ 8448.993715] LustreError: 270589:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8451.211183] LustreError: 270567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780243 with bad export cookie 2050788623304707250 [ 8451.230242] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8451.560292] Lustre: server umount lustre-MDT0001 complete [ 8463.157175] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 8465.966578] Key type lgssc unregistered [ 8466.283650] LNet: 274173:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8466.292442] LNetError: 274173:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8466.325722] LNet: Removed LNI 192.168.202.138@tcp [ 8467.127590] Key type .llcrypt unregistered [ 8467.129371] Key type ._llcrypt unregistered [ 8484.515434] Key type ._llcrypt registered [ 8484.518167] Key type .llcrypt registered [ 8484.620104] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 8485.424191] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8485.476209] alg: No test for adler32 (adler32-zlib) [ 8486.491132] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 8486.750125] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 8488.519285] Key type lgssc registered [ 8489.893963] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8500.504493] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8500.529566] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8516.191621] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8521.312398] LustreError: 275382:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.138@tcp: failed processing log, type 4: rc = -110 [ 8547.999237] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8554.402684] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8562.822495] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8565.921425] Lustre: server umount lustre-OST0000 complete [ 8584.573498] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 8587.379670] Key type lgssc unregistered [ 8587.661067] LNet: 277012:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8587.667457] LNetError: 277012:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8587.686609] LNet: Removed LNI 192.168.202.138@tcp [ 8588.507874] Key type .llcrypt unregistered [ 8588.510458] Key type ._llcrypt unregistered [ 8630.672885] Key type ._llcrypt registered [ 8630.675900] Key type .llcrypt registered [ 8630.779807] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 8631.770249] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8631.863968] alg: No test for adler32 (adler32-zlib) [ 8632.846280] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 8633.131516] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 8634.783809] Key type lgssc registered [ 8636.310990] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8651.282249] loop0: detected capacity change from 0 to 204800000 [ 8651.507233] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8651.513477] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8651.799390] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8652.874659] loop0: detected capacity change from 0 to 204800000 [ 8653.035870] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8653.052483] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8653.436985] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8654.362227] loop0: detected capacity change from 0 to 204800000 [ 8654.526522] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8654.539965] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8654.710040] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8655.816483] loop0: detected capacity change from 0 to 204800000 [ 8656.002510] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8657.033533] loop0: detected capacity change from 0 to 204800000 [ 8657.067820] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8661.486332] loop0: detected capacity change from 0 to 204800000 [ 8661.504986] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8665.775684] loop0: detected capacity change from 0 to 204800000 [ 8665.812453] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8670.179174] loop0: detected capacity change from 0 to 204800000 [ 8670.239383] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8674.101502] loop0: detected capacity change from 0 to 204800000 [ 8674.184898] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8674.254846] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8674.353875] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8674.366213] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8674.375542] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8674.383832] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8674.392146] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8674.413775] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8674.430296] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8674.468658] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8674.485154] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8674.497329] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8674.508090] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8674.531312] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8674.551936] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8674.584410] Lustre: 280456:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8680.360624] Lustre: server umount MGS complete [ 8681.184236] loop0: detected capacity change from 0 to 204800000 [ 8681.303041] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8681.844380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8681.883223] Lustre: lustre-MDT0000: Aborting client recovery [ 8682.789567] loop1: detected capacity change from 0 to 204800000 [ 8682.974255] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8683.086167] Lustre: 280889:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8683.096363] Lustre: 280889:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8683.104716] Lustre: 280889:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8683.110724] Lustre: 280889:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8683.119195] Lustre: 280889:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8683.126733] Lustre: 280889:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8683.138645] Lustre: 280889:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8683.160094] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8683.308467] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8683.361160] Lustre: lustre-MDT0001: Aborting client recovery [ 8684.112812] loop2: detected capacity change from 0 to 204800000 [ 8684.291344] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8684.369149] Lustre: 280970:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8684.385603] Lustre: 280970:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8684.397395] Lustre: 280970:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8684.404914] Lustre: 280970:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8684.415170] Lustre: 280970:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8684.422761] Lustre: 280970:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8684.431897] Lustre: 280970:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8684.518030] Lustre: Found index 0 for lustre-OST0000, updating log [ 8684.542595] Lustre: 280988:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8684.577320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8684.597763] Lustre: lustre-OST0000: Aborting recovery [ 8685.487924] loop3: detected capacity change from 0 to 204800000 [ 8685.687139] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8685.774099] Lustre: 281053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8685.800110] Lustre: 281053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8685.817513] Lustre: 281053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8685.836485] Lustre: 281053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8685.854490] Lustre: 281053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8685.864774] Lustre: 281053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8685.873805] Lustre: 281053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8685.895580] Lustre: Found index 1 for lustre-OST0001, updating log [ 8685.927558] Lustre: 281058:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8685.995868] Lustre: lustre-OST0001: Aborting recovery [ 8686.968241] Lustre: lustre-MDT0000: trigger OI scrub by RPC for [0x200000402:0x2:0x0]/0 with flags 0x4a: rc = 0 [ 8691.234118] LustreError: 280832:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8692.199173] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8692.203277] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8692.261285] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8692.318519] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8693.222260] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8693.224521] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8695.275679] LustreError: 280799:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 624us: rc = -2 [ 8695.779528] LustreError: 281065:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 692us: rc = -2 [ 8695.812797] LustreError: 281065:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 18 previous similar messages [ 8709.090404] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8709.102925] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8709.126140] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8713.696120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8713.703699] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8713.719314] Lustre: Skipped 1 previous similar message [ 8715.086211] Lustre: server umount lustre-MDT0000 complete [ 8716.140332] LustreError: 280781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780508 with bad export cookie 1659438788025121066 [ 8716.142528] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8716.148919] LustreError: 280781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8718.823161] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8718.830138] LustreError: 280796:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8718.830930] Lustre: Skipped 2 previous similar messages [ 8718.832453] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8718.832464] Lustre: Skipped 3 previous similar messages [ 8718.883786] LustreError: 280796:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8721.824500] LustreError: 280798:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8721.825505] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8721.852192] Lustre: Skipped 2 previous similar messages [ 8722.770433] Lustre: server umount lustre-MDT0001 complete [ 8738.271399] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8738.378240] Lustre: server umount lustre-OST0000 complete [ 8745.440223] Lustre: server umount lustre-OST0001 complete [ 8755.474414] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 11:02:26 (1790780546) [ 8780.240509] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 8820.008426] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8820.280561] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8820.310832] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8820.403577] Lustre: lustre-MDT0000: new disk, initializing [ 8820.509693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8820.519494] Lustre: Skipped 1 previous similar message [ 8820.542454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8825.150578] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8832.244228] Lustre: server umount lustre-MDT0000 complete [ 8850.303982] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:04:01 (1790780641) [ 8880.619780] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 8891.564616] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 8931.542385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8931.846848] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8931.931368] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8931.943819] Lustre: Skipped 1 previous similar message [ 8932.025471] Lustre: lustre-MDT0000: new disk, initializing [ 8932.098207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8932.113351] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8936.893979] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8950.387982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8950.504758] Lustre: 288160:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8950.583645] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8950.593626] Lustre: Skipped 1 previous similar message [ 8950.713742] Lustre: lustre-MDT0001: new disk, initializing [ 8950.851811] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8950.923353] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8950.943445] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8957.560796] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 8968.622384] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8974.572358] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8981.977801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8982.313601] Lustre: lustre-OST0000: new disk, initializing [ 8982.316309] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8982.321945] Lustre: 290132:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8982.425758] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8987.726870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8987.740387] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8987.832516] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8988.919836] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9000.696953] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9007.600532] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9022.384503] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9030.082961] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9075.171468] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9075.174992] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9075.179246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9075.179537] Lustre: Skipped 1 previous similar message [ 9075.183411] Lustre: Skipped 1 previous similar message [ 9076.532405] Lustre: server umount lustre-OST0000 complete [ 9083.872528] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9083.893758] LustreError: Skipped 1 previous similar message [ 9083.896662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9083.911608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9086.900467] Lustre: server umount lustre-MDT0000 complete [ 9090.089710] LustreError: 288153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780882 with bad export cookie 1659438788025124034 [ 9090.091849] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9090.103721] LustreError: 288153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9090.436679] Lustre: server umount lustre-MDT0001 complete [ 9099.478653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9104.303206] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9108.782994] Lustre: server umount MGS complete [ 9120.166859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9120.653109] LustreError: 294095:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9120.686345] LustreError: 294095:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9120.691500] LustreError: 294108:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9120.780976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9125.862066] LustreError: 294096:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9125.949344] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9129.964547] LustreError: 294095:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9133.839167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9138.367533] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9144.977340] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9151.236764] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9158.171316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9165.093097] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9171.917752] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9200.622687] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9200.646922] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9200.654262] Lustre: Skipped 1 previous similar message [ 9200.665651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9200.670143] Lustre: Skipped 2 previous similar messages [ 9204.893921] Lustre: server umount lustre-OST0000 complete [ 9210.853618] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9210.865643] LustreError: Skipped 1 previous similar message [ 9215.138222] Lustre: server umount lustre-MDT0000 complete [ 9216.993887] LustreError: 294096:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9217.024081] LustreError: 294096:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9218.836108] LustreError: 296818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790781011 with bad export cookie 1659438788025125868 [ 9218.838059] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9218.847775] LustreError: 296818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9219.193639] Lustre: server umount lustre-MDT0001 complete [ 9230.567955] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 9233.431585] Key type lgssc unregistered [ 9233.733586] LNet: 298036:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9233.741111] LNetError: 298036:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9234.796994] LNet: Removed LNI 192.168.202.138@tcp [ 9235.775892] Key type .llcrypt unregistered [ 9235.780415] Key type ._llcrypt unregistered [ 9271.872482] Key type ._llcrypt registered [ 9271.874944] Key type .llcrypt registered [ 9271.959679] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 9291.149815] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 9293.007151] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9293.188300] alg: No test for adler32 (adler32-zlib) [ 9294.300310] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 9294.519744] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 9296.223480] Key type lgssc registered [ 9297.835781] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9341.271430] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9341.353324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9342.674386] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9342.706492] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9342.773239] Lustre: lustre-MDT0000: new disk, initializing [ 9342.839270] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9342.859991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9347.597139] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9357.276734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9357.396995] Lustre: 302030:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9357.452408] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9357.456593] Lustre: Skipped 1 previous similar message [ 9357.515917] Lustre: lustre-MDT0001: new disk, initializing [ 9357.602883] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9357.647934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9357.661930] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9361.976380] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9370.915314] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9377.033678] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9384.405949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9384.676142] Lustre: lustre-OST0000: new disk, initializing [ 9384.679880] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9384.693304] Lustre: 303998:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9384.811483] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9390.768740] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9393.227719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9393.242995] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9393.403278] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9400.084421] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9408.695727] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9420.363805] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9431.890418] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9442.471672] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9490.915994] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9490.923769] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9490.937818] LustreError: Skipped 1 previous similar message [ 9490.952113] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9490.971669] Lustre: Skipped 1 previous similar message [ 9490.999064] Lustre: Skipped 1 previous similar message [ 9493.083738] Lustre: server umount lustre-OST0000 complete [ 9501.154048] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9501.157116] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9501.158811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9501.192379] Lustre: Skipped 1 previous similar message [ 9503.881430] Lustre: server umount lustre-MDT0000 complete [ 9508.619986] LustreError: 302023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790781300 with bad export cookie 15490665718262936632 [ 9508.626264] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9508.631695] LustreError: 302023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9508.963894] Lustre: server umount lustre-MDT0001 complete [ 9518.737483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9523.677606] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9528.126394] Lustre: server umount MGS complete [ 9540.126413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9540.879483] LustreError: 307919:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9540.895731] LustreError: 307919:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9540.899667] LustreError: 307932:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9540.989517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9545.725608] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9546.211116] LustreError: 307920:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9551.333301] LustreError: 307919:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9555.471059] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9555.919789] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9560.819962] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9568.230886] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9573.881322] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9580.866684] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9581.200917] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9587.517697] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9595.136403] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9627.617993] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9627.635391] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9627.643335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9627.652946] Lustre: Skipped 2 previous similar messages [ 9632.740754] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9632.750470] Lustre: Skipped 1 previous similar message [ 9633.219266] Lustre: server umount lustre-OST0000 complete [ 9642.977620] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9642.977816] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9642.992953] Lustre: Skipped 1 previous similar message [ 9642.994292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9646.737141] Lustre: server umount lustre-MDT0000 complete [ 9648.102811] LustreError: 307919:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9648.120186] LustreError: 307919:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9651.601814] LustreError: 307900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790781443 with bad export cookie 15490665718262938592 [ 9651.613475] LustreError: 307900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9651.614336] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9651.971516] Lustre: server umount lustre-MDT0001 complete [ 9665.570209] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 9669.218979] Key type lgssc unregistered [ 9669.622669] LNet: 311868:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9669.630942] LNetError: 311868:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9669.656081] LNet: Removed LNI 192.168.202.138@tcp [ 9670.717239] Key type .llcrypt unregistered [ 9670.718974] Key type ._llcrypt unregistered [ 9693.147238] Key type ._llcrypt registered [ 9693.151456] Key type .llcrypt registered [ 9693.288288] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 9708.349220] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 9710.064320] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9710.254720] alg: No test for adler32 (adler32-zlib) [ 9711.330502] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [ 9711.630280] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 9713.335260] Key type lgssc registered [ 9714.724793] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9741.152640] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 9756.416758] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9756.459238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9757.731822] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9757.753463] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9757.885772] Lustre: lustre-MDT0000: new disk, initializing [ 9758.006846] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9758.020624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9763.973594] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 9775.579103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9775.957290] Lustre: lustre-OST0000: new disk, initializing [ 9775.964956] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9775.974790] Lustre: Skipped 1 previous similar message [ 9775.985145] Lustre: 315933:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9776.106936] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9776.575283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9776.594250] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9776.641571] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9783.647286] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10625.654324] Lustre: server umount lustre-MDT0000 complete [10629.647406] LustreError: 315120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782421 with bad export cookie 1448544049491370295 [10629.655630] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10639.133410] Lustre: server umount lustre-OST0000 complete [10666.833862] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [10676.655859] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [10714.072087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10714.477484] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10714.529957] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10714.757782] Lustre: lustre-MDT0000: new disk, initializing [10714.884369] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10714.909375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10719.890605] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10731.382234] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10731.520117] Lustre: 320266:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10731.566520] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10731.568942] Lustre: Skipped 1 previous similar message [10731.635197] Lustre: lustre-MDT0001: new disk, initializing [10731.751249] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10731.800953] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10731.814541] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10736.888314] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10746.918342] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10752.153893] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10758.326570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10758.536342] Lustre: lustre-OST0000: new disk, initializing [10758.540588] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10758.545923] Lustre: 322235:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10758.647873] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10764.667572] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10768.480449] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10768.497515] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10768.609236] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10773.932847] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10779.760326] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10779.943336] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10784.382110] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10784.577823] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10789.344216] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10789.351694] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10789.358115] LustreError: Skipped 1 previous similar message [10789.363799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10789.372484] Lustre: Skipped 1 previous similar message [10789.412173] Lustre: Skipped 1 previous similar message [10792.576209] Lustre: server umount lustre-OST0000 complete [10798.566524] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10798.576329] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10798.583640] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10799.585486] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10799.587501] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10799.637144] Lustre: Skipped 1 previous similar message [10802.677165] Lustre: server umount lustre-MDT0000 complete [10806.264298] LustreError: 321070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782598 with bad export cookie 1448544049494311114 [10806.268162] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10806.280221] LustreError: 321070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10806.562108] Lustre: server umount lustre-MDT0001 complete [10810.298308] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10811.846059] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:36:43 (1790782603) [10818.241623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10818.778585] LustreError: 324547:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10818.804373] LustreError: 324547:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10818.887282] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10823.774286] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10824.161856] LustreError: 324548:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10829.288558] LustreError: 324547:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10832.233180] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10832.683411] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10837.515271] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10845.045715] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10850.636208] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10857.945125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10858.339431] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10864.826958] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10872.162851] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10879.040425] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10879.244379] Lustre: lustre-OST0001: new disk, initializing [10879.271816] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10879.289874] Lustre: 327384:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10879.427909] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10886.144514] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [10886.247205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10886.257058] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10886.310808] LustreError: 327744:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10886.323085] LustreError: 327744:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10887.333482] LustreError: 327750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10888.353535] LustreError: 327750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10888.366594] LustreError: 327750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10890.476189] LustreError: 327750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10890.503252] LustreError: 327750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [10894.568130] LustreError: 327744:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10894.575429] LustreError: 327744:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10896.184079] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10896.447861] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10902.548912] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c9e07774000.ost_server_uuid 50 [10902.824050] LustreError: 327744:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10902.832396] LustreError: 327744:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10904.409589] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c9e07774000.ost_server_uuid in FULL state after 0 sec [10910.119165] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c9e07774000.ost_server_uuid 50 [10911.774506] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9c9e07774000.ost_server_uuid in FULL state after 0 sec [10914.171733] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10927.605268] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10927.611451] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10927.623891] Lustre: Skipped 1 previous similar message [10932.523796] Lustre: lustre-OST0001: Not available for connect from 192.168.202.38@tcp (stopping) [10932.532375] Lustre: Skipped 1 previous similar message [10933.747573] Lustre: server umount lustre-OST0001 complete [10937.577803] LustreError: 326368:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10941.411600] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10941.418813] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10941.430060] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10941.443475] Lustre: Skipped 2 previous similar messages [10942.951094] LustreError: 326369:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10942.962617] LustreError: 326369:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10946.062385] Lustre: server umount lustre-OST0000 complete [10950.625235] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10950.638301] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10950.664557] Lustre: Skipped 1 previous similar message [10956.356321] Lustre: server umount lustre-MDT0000 complete [10958.304717] LustreError: 324542:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10958.318566] LustreError: 324542:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10960.500520] LustreError: 325283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782752 with bad export cookie 1448544049494311996 [10960.514869] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10960.519934] LustreError: 325283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10960.902366] Lustre: server umount lustre-MDT0001 complete [10971.653288] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [10974.229646] Key type lgssc unregistered [10974.565556] LNet: 330071:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10974.574081] LNetError: 330071:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10974.606017] LNet: Removed LNI 192.168.202.138@tcp [10975.567130] Key type .llcrypt unregistered [10975.569973] Key type ._llcrypt unregistered [11001.394813] Key type ._llcrypt registered [11001.398607] Key type .llcrypt registered [11001.517316] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [11002.970527] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11003.059550] alg: No test for adler32 (adler32-zlib) [11004.258280] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [11004.704996] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [11006.431523] Key type lgssc registered [11008.190821] Lustre: Echo OBD driver; http://www.lustre.org/ [11017.443547] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11017.457333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11018.931220] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11024.151348] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11029.477880] LustreError: 331513:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11029.498391] LustreError: 331513:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11031.923284] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11034.594598] LustreError: 331514:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11038.512858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11038.863942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11043.256553] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11052.531257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11052.933939] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11059.941579] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11064.184528] LustreError: 333753:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11064.209265] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11064.216374] LustreError: 333753:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11067.320720] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11069.419413] LustreError: 333191:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11069.445703] LustreError: 333191:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11074.375038] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11074.578188] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11079.744424] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11082.200337] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11090.399964] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11096.771911] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11096.950875] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11202.528558] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11202.538598] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11202.564715] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11202.584059] Lustre: Skipped 1 previous similar message [11204.436746] Lustre: server umount lustre-OST0001 complete [11207.655532] LustreError: 333194:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11207.680969] LustreError: 333194:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11212.774793] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11212.786692] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11212.810539] Lustre: Skipped 1 previous similar message [11212.814789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11212.834187] Lustre: Skipped 2 previous similar messages [11216.701229] Lustre: server umount lustre-OST0000 complete [11223.008356] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11223.012574] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11223.013846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11223.039705] Lustre: Skipped 1 previous similar message [11223.069819] LustreError: Skipped 1 previous similar message [11228.128844] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11228.138680] Lustre: Skipped 2 previous similar messages [11233.250841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11233.257382] Lustre: Skipped 1 previous similar message [11236.321265] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11236.690853] Lustre: server umount lustre-MDT0000 complete [11238.379140] LustreError: 333922:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11238.396719] LustreError: 333922:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11240.613875] LustreError: 331495:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783032 with bad export cookie 5744555068444336735 [11240.615889] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11240.620899] LustreError: 331495:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11241.085872] Lustre: server umount lustre-MDT0001 complete [11251.748963] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [11254.701802] Key type lgssc unregistered [11254.976641] LNet: 336457:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11254.982959] LNetError: 336457:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11255.009247] LNet: Removed LNI 192.168.202.138@tcp [11255.654612] Key type .llcrypt unregistered [11255.656331] Key type ._llcrypt unregistered [11301.457895] Key type ._llcrypt registered [11301.464242] Key type .llcrypt registered [11301.623881] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [11302.809380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11302.831484] alg: No test for adler32 (adler32-zlib) [11303.955770] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [11304.215646] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [11305.943208] Key type lgssc registered [11307.695410] Lustre: Echo OBD driver; http://www.lustre.org/ [11318.294518] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11318.312472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11319.622945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11323.960946] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11329.007622] LustreError: 338765:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11329.019121] LustreError: 338765:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11332.633727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11332.877741] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11337.145386] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11339.948746] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11346.569434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11347.079134] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11353.329949] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11358.378140] LustreError: 340256:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11362.303234] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11362.553906] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11364.606418] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11364.608703] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11369.400845] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11380.692979] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11387.758026] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11400.675226] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11400.692914] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11400.700050] Lustre: Skipped 2 previous similar messages [11400.710486] Lustre: Skipped 3 previous similar messages [11405.804931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11405.814268] Lustre: Skipped 3 previous similar messages [11407.158666] Lustre: server umount lustre-MDT0000 complete [11410.922833] LustreError: 339290:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11410.946767] LustreError: 339290:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11411.529463] LustreError: 338746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783203 with bad export cookie 9637213428935722067 [11411.537094] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11411.541803] LustreError: 338746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11411.956272] Lustre: server umount lustre-MDT0001 complete [11426.659118] Lustre: server umount lustre-OST0000 complete [11441.397132] Lustre: server umount lustre-OST0001 complete [11460.938313] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [11473.241894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11474.007583] LustreError: 344033:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11474.162531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11479.525238] LustreError: 344034:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11479.537895] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11489.749834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11489.764057] LustreError: 344033:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11489.787831] LustreError: 344033:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11490.206021] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11495.599831] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11499.387704] Lustre: 345173:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11506.853237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11507.229628] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11512.751462] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11515.437740] LustreError: 345527:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11519.548077] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11521.772984] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11527.669457] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11528.713694] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11536.141557] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11539.592750] Lustre: 347044:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11546.594549] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11546.607347] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11546.627212] Lustre: Skipped 1 previous similar message [11546.641871] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11548.156093] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11548.177776] Lustre: Skipped 2 previous similar messages [11549.513260] Lustre: server umount lustre-MDT0000 complete [11553.043399] LustreError: 344015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783345 with bad export cookie 9637213428935723768 [11553.052492] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11553.056352] LustreError: 344015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11553.248809] LustreError: 344029:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11553.249477] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11553.261395] LustreError: 344029:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11553.264616] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11553.264623] Lustre: Skipped 4 previous similar messages [11553.297909] Lustre: Skipped 1 previous similar message [11553.425826] Lustre: server umount lustre-MDT0001 complete [11559.247311] Lustre: server umount lustre-OST0000 complete [11563.498910] Lustre: server umount lustre-OST0001 complete [11566.682679] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11568.031682] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11569.630631] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:49:20 (1790783360) [11593.153800] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [11603.839146] loop0: detected capacity change from 0 to 1099511627776 [11604.683543] SGI XFS with ACLs, security attributes, quota, no debug enabled [11604.688493] *** VALIDATE xfs *** [11604.725877] XFS (loop0): Mounting V5 Filesystem [11604.761377] XFS (loop0): Ending clean mount [11606.523734] loop1: detected capacity change from 0 to 18691697672192 [11715.156511] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [11724.580651] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [11759.494269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11759.732128] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11759.751410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11759.817911] Lustre: lustre-MDT0000: new disk, initializing [11759.872299] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11759.876156] Lustre: Skipped 1 previous similar message [11759.886675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11765.058226] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11776.915941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11777.149896] Lustre: 352178:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11777.274468] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11777.279667] Lustre: Skipped 1 previous similar message [11777.408195] Lustre: lustre-MDT0001: new disk, initializing [11777.644070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11777.681710] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11783.606894] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11796.569475] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11802.727210] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11810.905580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11811.329560] Lustre: lustre-OST0000: new disk, initializing [11811.333647] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11811.339452] Lustre: 354147:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11812.856789] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11812.867373] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11813.049887] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11817.911803] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11827.154695] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11831.820887] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11832.022441] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11836.857880] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11837.141904] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11838.947734] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11838.956735] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11838.968529] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11838.976458] Lustre: Skipped 1 previous similar message [11843.553635] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11848.682079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11848.697714] Lustre: Skipped 3 previous similar messages [11853.279143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11853.506466] Lustre: server umount lustre-OST0000 complete [11858.913354] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11858.927279] Lustre: Skipped 1 previous similar message [11864.751693] Lustre: server umount lustre-MDT0000 complete [11869.153024] LustreError: 352189:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11869.190194] LustreError: 352189:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11870.393175] LustreError: 352171:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783662 with bad export cookie 9637213428935725322 [11870.402720] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11870.409832] LustreError: 352171:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11870.961804] Lustre: server umount lustre-MDT0001 complete [11881.584145] XFS (loop0): Unmounting Filesystem [11886.506242] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:54:37 (1790783677) [11894.394945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11895.042404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11895.045199] Lustre: Skipped 2 previous similar messages [11899.753562] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11908.912617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11914.607900] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11923.299195] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11929.316654] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11936.724201] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11944.684511] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [11952.783814] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11965.928124] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11965.934946] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11965.940388] Lustre: Skipped 1 previous similar message [11965.952290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11965.956648] Lustre: Skipped 4 previous similar messages [11971.997285] Lustre: server umount lustre-OST0000 complete [11983.253580] Lustre: server umount lustre-MDT0000 complete [11986.572946] LustreError: 356686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783778 with bad export cookie 9637213428935726204 [11986.582779] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11986.844961] Lustre: server umount lustre-MDT0001 complete [11997.166712] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [11999.851358] Key type lgssc unregistered [12000.124173] LNet: 360336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12000.140685] LNetError: 360336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12000.171041] LNet: Removed LNI 192.168.202.138@tcp [12001.088152] Key type .llcrypt unregistered [12001.090466] Key type ._llcrypt unregistered [12028.630313] Key type ._llcrypt registered [12028.633769] Key type .llcrypt registered [12028.744098] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [12029.883412] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12030.019583] alg: No test for adler32 (adler32-zlib) [12031.112772] Lustre: Lustre: Build Version: 2.17.59_3_gf57cca5 [12031.446958] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [12033.151330] Key type lgssc registered [12034.388211] Lustre: Echo OBD driver; http://www.lustre.org/ [12042.557537] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [12042.598873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12044.234259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12049.235765] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [12054.497901] LustreError: 361921:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12054.517814] LustreError: 361921:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12058.750818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12059.289828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12065.000153] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [12074.551760] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12081.033579] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12089.178753] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12089.321923] LustreError: 363745:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12089.570419] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12096.544282] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [12105.427451] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12115.950492] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12115.961155] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12115.998257] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12118.796269] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [12119.544241] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12121.546677] Lustre: server umount lustre-MDT0001 complete [12123.904105] LustreError: 361917:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12129.009042] LustreError: 361916:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12129.041722] LustreError: 361916:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12132.547529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12132.655252] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12132.658931] Lustre: Found index 1 for lustre-MDT0001, updating log [12132.705742] Lustre: 361908:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12133.157775] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12137.781952] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [12138.492428] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12140.557460] LustreError: 365523:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12140.569780] LustreError: 365523:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12140.587544] LustreError: 361911:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12140.700277] LustreError: 365526:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.138@tcp: cfg command failed: rc = -17 [12140.711457] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12140.711457] [12446.481633] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [12446.494023] Lustre: Skipped 2 previous similar messages [12450.798207] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12450.798725] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12450.828449] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12450.837355] Lustre: Skipped 1 previous similar message [12450.858047] Lustre: Skipped 1 previous similar message [12452.741363] Lustre: server umount lustre-MDT0001 complete [12455.924257] LustreError: 361916:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12455.934105] LustreError: 361916:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12466.155545] LustreError: 362649:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12466.178328] LustreError: 362649:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [12467.894480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12468.046636] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12468.054059] Lustre: Found index 1 for lustre-MDT0001, updating log [12468.105581] Lustre: 361907:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12468.809027] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12473.877101] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12475.014670] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [12475.412033] LustreError: 373625:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12475.417135] LustreError: 373625:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12475.421503] LustreError: 361911:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12475.424400] LustreError: 361911:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12475.465285] LustreError: 373630:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12475.470587] LustreError: 373630:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12475.474857] LustreError: 373630:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.138@tcp: cfg command failed: rc = -17 [12475.481766] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.138@tcp [12475.481766] [12784.404288] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [12784.407885] Lustre: Skipped 1 previous similar message [12786.146733] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12786.149479] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12790.074273] Lustre: server umount lustre-MDT0001 complete [12791.287677] LustreError: 362649:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12791.316746] LustreError: 362649:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12801.450699] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12801.529485] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12801.539210] Lustre: Found index 1 for lustre-MDT0001, updating log [12801.590793] Lustre: 361907:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12802.063748] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12807.153850] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12807.701475] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [12811.263396] LustreError: 381662:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12811.274167] LustreError: 381662:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12811.295593] LustreError: 361911:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12811.298651] LustreError: 361911:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12811.418833] LustreError: 381673:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12811.460257] LustreError: 381673:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.138@tcp: cfg command failed: rc = -114 [12811.480815] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12811.480815] [13129.703798] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13129.705953] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13129.719134] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13129.719141] Lustre: Skipped 3 previous similar messages [13129.753511] Lustre: Skipped 3 previous similar messages [13135.833296] Lustre: server umount lustre-MDT0000 complete [13139.369097] LustreError: 362656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790784931 with bad export cookie 16164281140126363748 [13139.376941] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13139.383725] LustreError: 362656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13139.936581] LustreError: 365527:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13139.937994] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13139.959495] LustreError: 365527:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [13145.866793] Lustre: server umount lustre-MDT0001 complete [13149.872305] Lustre: server umount lustre-OST0000 complete [13159.865279] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [13170.045737] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [13209.467179] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:16:40 (1790785000) [13217.311979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13217.596210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13217.630776] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13217.734126] Lustre: lustre-MDT0000: new disk, initializing [13217.822281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13217.839588] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13223.279992] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [13235.937112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13236.131199] Lustre: 392066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13236.240029] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13236.242757] Lustre: Skipped 1 previous similar message [13236.423400] Lustre: lustre-MDT0001: new disk, initializing [13236.712285] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13236.780849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13236.793356] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13242.175197] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [13253.372888] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13258.509870] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13267.095490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13267.333946] Lustre: lustre-OST0000: new disk, initializing [13267.338058] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13267.343710] Lustre: 394037:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13267.444858] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13273.350430] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [13273.705668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13273.725978] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13273.811048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13282.833878] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13292.012736] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13292.027659] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13292.048811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13292.052823] Lustre: Skipped 7 previous similar messages [13292.666355] Lustre: server umount lustre-MDT0000 complete [13294.581450] LustreError: 392077:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13294.616913] LustreError: 392077:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [13296.742325] LustreError: 392057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790785089 with bad export cookie 16164281140126365701 [13296.758663] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13296.763701] LustreError: 392057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13296.763906] LustreError: 392874:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13296.764069] LustreError: 392874:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000001fbe707a x1877772938377216/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [13297.128737] Lustre: server umount lustre-MDT0001 complete [13311.521338] Lustre: server umount lustre-OST0000 complete [13343.036936] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [13353.312139] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [13387.754683] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13101 sec ========= 12:19:38 (1790785178) [13389.646689] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:19:40 (1790785180) === [13393.712298] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:19:44 (1790785184) === [13434.126956] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [13437.700281] Key type lgssc unregistered [13438.022594] LNet: 400401:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13438.034134] LNetError: 400401:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13439.084567] LNet: Removed LNI 192.168.202.138@tcp [13440.185098] Key type .llcrypt unregistered [13440.188185] Key type ._llcrypt unregistered