[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 459342010 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003307] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005017] kvm-guest: setup PV IPIs [ 0.008809] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010014] pid_max: default: 32768 minimum: 301 [ 0.012099] LSM: Security Framework initializing [ 0.013054] Yama: becoming mindful. [ 0.014045] SELinux: Initializing. [ 0.015098] *** VALIDATE selinux *** [ 0.022633] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026778] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027135] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028116] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029115] *** VALIDATE tmpfs *** [ 0.031351] *** VALIDATE proc *** [ 0.032220] *** VALIDATE cgroup *** [ 0.033010] *** VALIDATE cgroup2 *** [ 0.034249] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035175] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037030] Spectre V2 : User space: Vulnerable [ 0.038006] Speculative Store Bypass: Vulnerable [ 0.041181] debug: unmapping init [mem 0xffffffff91259000-0xffffffff91260fff] [ 0.043167] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044744] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045026] ... version: 2 [ 0.046014] ... bit width: 48 [ 0.047015] ... generic registers: 4 [ 0.048015] ... value mask: 0000ffffffffffff [ 0.049014] ... max period: 00007fffffffffff [ 0.050016] ... fixed-purpose events: 3 [ 0.051015] ... event mask: 000000070000000f [ 0.052291] rcu: Hierarchical SRCU implementation. [ 0.054448] smp: Bringing up secondary CPUs ... [ 0.055606] x86: Booting SMP configuration: [ 0.056027] .... node #0, CPUs: #1 #2 #3 [ 0.059484] smp: Brought up 1 node, 4 CPUs [ 0.061016] smpboot: Max logical packages: 1 [ 0.062020] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.132031] node 0 deferred pages initialised in 67ms [ 0.134111] devtmpfs: initialized [ 0.136207] x86/mm: Memory block size: 128MB [ 0.139886] gcov: version magic: 0x41383552 [ 0.144181] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.145093] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.146290] pinctrl core: initialized pinctrl subsystem [ 0.147215] [ 0.147835] ************************************************************* [ 0.148017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.149011] ** ** [ 0.150014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.151014] ** ** [ 0.152015] ** This means that this kernel is built to expose internal ** [ 0.153014] ** IOMMU data structures, which may compromise security on ** [ 0.154016] ** your system. ** [ 0.155013] ** ** [ 0.156013] ** If you see this message and you are not debugging the ** [ 0.157012] ** kernel, report this immediately to your vendor! ** [ 0.158013] ** ** [ 0.159013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.160015] ************************************************************* [ 0.161868] NET: Registered protocol family 16 [ 0.163386] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.165115] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.168068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.172453] cpuidle: using governor menu [ 0.174810] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.178497] PCI: Using configuration type 1 for base access [ 0.181125] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.192043] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.194015] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.197085] cryptd: max_cpu_qlen set to 1000 [ 0.199225] ACPI: Added _OSI(Module Device) [ 0.201012] ACPI: Added _OSI(Processor Device) [ 0.202008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.203020] ACPI: Added _OSI(Processor Aggregator Device) [ 0.206959] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.213400] ACPI: Interpreter enabled [ 0.214052] ACPI: PM: (supports S0 S3 S4 S5) [ 0.215010] ACPI: Using IOAPIC for interrupt routing [ 0.217096] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.220360] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.226884] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.229078] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.232031] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.234083] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.238971] acpiphp: Slot [2] registered [ 0.240107] acpiphp: Slot [5] registered [ 0.241075] acpiphp: Slot [6] registered [ 0.242080] acpiphp: Slot [7] registered [ 0.244097] acpiphp: Slot [8] registered [ 0.245117] acpiphp: Slot [9] registered [ 0.246096] acpiphp: Slot [10] registered [ 0.247088] acpiphp: Slot [3] registered [ 0.249110] acpiphp: Slot [4] registered [ 0.250075] acpiphp: Slot [11] registered [ 0.251075] acpiphp: Slot [12] registered [ 0.253078] acpiphp: Slot [13] registered [ 0.254122] acpiphp: Slot [14] registered [ 0.255080] acpiphp: Slot [15] registered [ 0.257084] acpiphp: Slot [16] registered [ 0.258070] acpiphp: Slot [17] registered [ 0.259075] acpiphp: Slot [18] registered [ 0.260075] acpiphp: Slot [19] registered [ 0.262099] acpiphp: Slot [20] registered [ 0.263072] acpiphp: Slot [21] registered [ 0.264085] acpiphp: Slot [22] registered [ 0.266097] acpiphp: Slot [23] registered [ 0.267095] acpiphp: Slot [24] registered [ 0.269109] acpiphp: Slot [25] registered [ 0.270117] acpiphp: Slot [26] registered [ 0.272079] acpiphp: Slot [27] registered [ 0.273080] acpiphp: Slot [28] registered [ 0.274083] acpiphp: Slot [29] registered [ 0.276081] acpiphp: Slot [30] registered [ 0.277080] acpiphp: Slot [31] registered [ 0.278053] PCI host bridge to bus 0000:00 [ 0.279016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.281020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.284023] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.286019] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.289072] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.292025] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.294167] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.297647] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.301337] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.310016] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.314813] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.317015] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.319015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.321021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.323453] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.327828] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.330080] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.332480] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.337013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.347642] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.351065] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.355000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.360015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.366016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.380014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.390065] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.396029] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.404017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.431017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.450994] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.473016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.486017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.510020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.530372] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.546019] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.560018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.581019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.592985] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.611016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.620026] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.655017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.664430] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.674017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.682016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.705018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.718053] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.724424] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.727434] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.731357] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.734196] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.739326] iommu: Default domain type: Passthrough [ 0.741448] SCSI subsystem initialized [ 0.743136] ACPI: bus type USB registered [ 0.744125] usbcore: registered new interface driver usbfs [ 0.746082] usbcore: registered new interface driver hub [ 0.748090] usbcore: registered new device driver usb [ 0.750258] pps_core: LinuxPPS API ver. 1 registered [ 0.752013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.755065] PTP clock support registered [ 0.757140] EDAC MC: Ver: 3.0.0 [ 0.759487] PCI: Using ACPI for IRQ routing [ 0.763081] NetLabel: Initializing [ 0.764011] NetLabel: domain hash size = 128 [ 0.765006] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.766071] NetLabel: unlabeled traffic allowed by default [ 0.768100] vgaarb: loaded [ 0.770169] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.772013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.777393] clocksource: Switched to clocksource kvm-clock [ 0.884904] VFS: Disk quotas dquot_6.6.0 [ 0.886668] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.889334] *** VALIDATE ramfs *** [ 0.891255] *** VALIDATE hugetlbfs *** [ 0.894209] pnp: PnP ACPI init [ 0.897211] pnp: PnP ACPI: found 6 devices [ 0.914821] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.919198] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.921490] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.923826] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.926490] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.928868] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.931609] NET: Registered protocol family 2 [ 0.934084] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.938407] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.941799] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.946904] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.950185] TCP: Hash tables configured (established 65536 bind 65536) [ 0.953056] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.956076] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.958520] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.961527] NET: Registered protocol family 1 [ 0.964180] RPC: Registered named UNIX socket transport module. [ 0.966512] RPC: Registered udp transport module. [ 0.968308] RPC: Registered tcp transport module. [ 0.970130] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.972623] NET: Registered protocol family 44 [ 0.974464] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.976776] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.978131] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.980141] PCI: CLS 0 bytes, default 64 [ 0.982172] Unpacking initramfs... [ 2.361772] debug: unmapping init [mem 0xffff99bf3cc54000-0xffff99bf3ffbffff] [ 2.366885] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.370230] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.374597] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.895039] Initialise system trusted keyrings [ 2.897494] Key type blacklist registered [ 2.899940] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.909743] zbud: loaded [ 2.912975] *** VALIDATE nfs *** [ 2.914861] *** VALIDATE nfs4 *** [ 2.917124] pstore: using deflate compression [ 2.921720] Platform Keyring initialized [ 3.018712] NET: Registered protocol family 38 [ 3.020669] Key type asymmetric registered [ 3.022182] Asymmetric key parser 'x509' registered [ 3.023901] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.027125] io scheduler mq-deadline registered [ 3.029151] io scheduler kyber registered [ 3.031142] io scheduler bfq registered [ 3.034058] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.037591] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.040955] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.044321] ACPI: Power Button [PWRF] [ 3.050184] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.056850] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.081834] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.094375] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.128562] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.158620] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.189596] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.194386] Non-volatile memory driver v1.3 [ 3.195929] Linux agpgart interface v0.103 [ 3.229657] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.231826] vda: detected capacity change from 0 to 74698752 [ 3.265099] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.268486] vdb: detected capacity change from 0 to 1073741824 [ 3.295617] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.299848] vdc: detected capacity change from 0 to 2621440000 [ 3.321902] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.324930] vdd: detected capacity change from 0 to 2621440000 [ 3.346425] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.349103] vde: detected capacity change from 0 to 4294967296 [ 3.371790] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.374148] vdf: detected capacity change from 0 to 4294967296 [ 3.388693] libphy: Fixed MDIO Bus: probed [ 3.398111] usbcore: registered new interface driver usbserial_generic [ 3.400207] usbserial: USB Serial support registered for generic [ 3.402207] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.405689] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.407325] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.409525] mousedev: PS/2 mouse device common for all mice [ 3.411618] rtc_cmos 00:05: RTC can wake from S4 [ 3.412386] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.414490] rtc_cmos 00:05: registered as rtc0 [ 3.418551] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.421817] intel_pstate: CPU model not supported [ 3.421931] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.427212] hid: raw HID events driver (C) Jiri Kosina [ 3.429389] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.430125] usbcore: registered new interface driver usbhid [ 3.435584] usbhid: USB HID core driver [ 3.437920] drop_monitor: Initializing network drop monitor service [ 3.441562] Initializing XFRM netlink socket [ 3.444151] NET: Registered protocol family 10 [ 3.447595] Segment Routing with IPv6 [ 3.449449] NET: Registered protocol family 17 [ 3.452180] mpls_gso: MPLS GSO support [ 3.457210] RAS: Correctable Errors collector initialized. [ 3.459467] AVX version of gcm_enc/dec engaged. [ 3.460983] AES CTR mode by8 optimization enabled [ 3.518569] sched_clock: Marking stable (3518550499, 0)->(4456416178, -937865679) [ 3.522084] registered taskstats version 1 [ 3.523803] Loading compiled-in X.509 certificates [ 3.525547] zswap: loaded using pool lzo/zbud [ 3.543738] Key type big_key registered [ 3.553067] Key type encrypted registered [ 3.554658] ima: No TPM chip found, activating TPM-bypass! [ 3.556543] ima: Allocated hash algorithm: sha1 [ 3.558228] ima: No architecture policies found [ 3.559922] evm: Initialising EVM extended attributes: [ 3.561557] evm: security.selinux [ 3.562471] evm: security.ima [ 3.563211] evm: security.capability [ 3.564142] evm: HMAC attrs: 0x1 [ 3.565976] rtc_cmos 00:05: setting system clock to 2026-09-03 13:20:50 UTC (1788441650) [ 3.569957] debug: unmapping init [mem 0xffffffff92203000-0xffffffff923fffff] [ 3.572018] debug: unmapping init [mem 0xffffffff90f82000-0xffffffff91258fff] [ 3.584075] Write protecting the kernel read-only data: 28672k [ 3.586067] debug: unmapping init [mem 0xffffffff8f603000-0xffffffff8f7fffff] [ 3.587606] debug: unmapping init [mem 0xffffffff8ff14000-0xffffffff8fffffff] [ 3.616921] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.625033] systemd[1]: Detected virtualization kvm. [ 3.626386] systemd[1]: Detected architecture x86-64. [ 3.627788] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.660184] systemd[1]: No hostname configured. [ 3.661458] systemd[1]: Set hostname to . [ 3.663023] random: systemd: uninitialized urandom read (16 bytes read) [ 3.665418] systemd[1]: Initializing machine ID from random generator. [ 3.704192] random: ln: uninitialized urandom read (6 bytes read) [ 3.785090] random: systemd: uninitialized urandom read (16 bytes read) [ 3.787929] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.792486] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.798747] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. [ OK ] Listening on udev Control Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.333365] device-mapper: uevent: version 1.0.3 [ 4.335660] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.003672] random: fast init done [ 5.012424] virtio_net virtio0 ens2: renamed from eth0 [ 5.056801] scsi host0: ata_piix [ 5.069422] scsi host1: ata_piix [ 5.070989] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.073637] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.084852] random: crng init done [ 10.089745] random: 7 urandom warning(s) missed due to ratelimiting [ 10.212992] dracut-initqueue[588]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 12.719348] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 16.202405] printk: systemd: 25 output lines suppressed due to ratelimiting [ 16.956442] SELinux: Disabled at runtime. [ 17.088958] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 17.106346] systemd[1]: Detected virtualization kvm. [ 17.108222] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 18.822671] systemd[1]: initrd-switch-root.service: Succeeded. [ 18.836528] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 18.844534] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 18.853943] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 18.860424] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 18.874197] systemd[1]: Starting Journal Service... Starting Journal Service... [ 18.885188] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. [ OK ] Created slice User and Session Slice. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Initrd Root File System. Mounting Huge Pages File System... Starting Remount Root and Kernel File Systems... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Kernel Debug File System... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ 19.401939] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 20.210263] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.477156] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.543905] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 22.023600] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 22.149128] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit)[ 26.517454] Key type dns_resolver registered [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit)[ 27.657464] NFS: Registering the id_resolver key type [ 27.664072] Key type id_resolver registered [ 27.665653] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Daily Cleanup of Temporary Directories. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg206-server login: [ 89.758344] libcfs: loading out-of-tree module taints kernel. [ 89.792213] Key type ._llcrypt registered [ 89.794435] Key type .llcrypt registered [ 89.941092] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 111.483837] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 113.155601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 113.168590] alg: No test for adler32 (adler32-zlib) [ 114.932753] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 116.232556] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 117.944240] Key type lgssc registered [ 120.387441] Lustre: Echo OBD driver; http://www.lustre.org/ [ 141.386973] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 167.540045] hrtimer: interrupt took 12288059 ns [ 189.248108] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 204.134293] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 204.204094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 205.630385] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 205.718811] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 205.818119] Lustre: lustre-MDT0000: new disk, initializing [ 205.917226] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 205.940228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 211.092334] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 227.298039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 227.510685] Lustre: 6511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 227.594916] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 227.605940] Lustre: Skipped 1 previous similar message [ 227.743235] Lustre: lustre-MDT0001: new disk, initializing [ 227.899118] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 228.027865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 228.046567] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 236.099150] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 242.224963] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 254.715084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 255.072845] Lustre: lustre-OST0000: new disk, initializing [ 255.075400] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 255.089943] Lustre: 8451:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 255.186293] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 260.719365] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 260.732472] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 260.848877] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 261.978451] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 279.663379] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 279.805725] Lustre: lustre-OST0001: new disk, initializing [ 279.809591] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 279.814692] Lustre: 9524:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 279.877660] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 287.281774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 287.303234] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 287.365217] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 287.369062] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 300.369543] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 306.342717] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 312.763619] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing check_logdir /tmp/testlogs/ [ 319.362144] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing yml_node [ 323.371810] Lustre: DEBUG MARKER: Client: 2.17.56.51 [ 326.578044] Lustre: DEBUG MARKER: MDS: 2.17.56.51 [ 329.127453] Lustre: DEBUG MARKER: OSS: 2.17.56.51 [ 331.593868] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 3 09:26:15 EDT 2026 [ 350.135848] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 352.211041] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 369.121460] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 369.127100] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 369.138728] Lustre: Skipped 1 previous similar message [ 369.158230] Lustre: Skipped 2 previous similar messages [ 374.243534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 374.278833] Lustre: Skipped 3 previous similar messages [ 374.883783] Lustre: server umount lustre-MDT0000 complete [ 378.896291] LustreError: 7927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442025 with bad export cookie 12780075451929799270 [ 378.900557] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 378.909902] LustreError: 7927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 379.360664] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 379.361627] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 379.377731] Lustre: Skipped 3 previous similar messages [ 379.398288] Lustre: Skipped 1 previous similar message [ 383.397573] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 383.408868] Lustre: Skipped 1 previous similar message [ 385.435418] Lustre: server umount lustre-MDT0001 complete [ 389.777119] Lustre: server umount lustre-OST0000 complete [ 393.734490] Lustre: server umount lustre-OST0001 complete [ 400.322378] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 410.893796] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 449.271316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 449.537337] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 449.568446] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 449.642459] Lustre: lustre-MDT0000: new disk, initializing [ 449.749992] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 449.772952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 454.961431] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 466.216907] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 466.326613] Lustre: 15074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 466.370785] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 466.373934] Lustre: Skipped 1 previous similar message [ 466.464402] Lustre: lustre-MDT0001: new disk, initializing [ 466.562678] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 466.605427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 466.610635] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 471.130387] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 479.677545] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 485.580428] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 493.387323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 493.838279] Lustre: lustre-OST0000: new disk, initializing [ 493.846692] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 493.854606] Lustre: 17044:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 493.969723] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 495.546571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 495.559822] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 495.685388] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 501.608558] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 511.833670] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 516.710722] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 516.895815] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 521.106539] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 521.332391] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 526.305466] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 526.317844] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 526.331323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 526.340483] Lustre: Skipped 1 previous similar message [ 526.358793] Lustre: Skipped 1 previous similar message [ 529.756249] Lustre: server umount lustre-OST0000 complete [ 536.550513] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 536.551518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 536.573942] Lustre: Skipped 1 previous similar message [ 536.588273] Lustre: Skipped 1 previous similar message [ 539.873539] Lustre: server umount lustre-MDT0000 complete [ 541.678796] LustreError: 15946:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.687084] LustreError: 15946:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 544.007735] LustreError: 15069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442190 with bad export cookie 12780075451929800810 [ 544.012764] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 544.026820] LustreError: 15069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 544.518924] Lustre: server umount lustre-MDT0001 complete [ 549.610775] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:29:54 (1788442194) [ 556.646772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 557.164580] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.288139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 562.349502] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 562.662889] LustreError: 19311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 567.784877] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 571.448951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 577.481807] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 585.497071] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 592.720342] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 600.171706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 607.958556] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 615.947369] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 622.915960] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 632.292687] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 632.296701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 632.320056] Lustre: Skipped 1 previous similar message [ 632.334544] Lustre: Skipped 2 previous similar messages [ 635.665161] Lustre: server umount lustre-MDT0000 complete [ 637.418463] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 637.428414] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 639.917156] LustreError: 19291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442286 with bad export cookie 12780075451929801692 [ 639.926561] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 639.931466] LustreError: 19291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 640.415098] Lustre: server umount lustre-MDT0001 complete [ 655.494329] Lustre: server umount lustre-OST0000 complete [ 675.227861] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 686.083153] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 686.360292] LustreError: 23696:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 686.409347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 686.414086] Lustre: Skipped 2 previous similar messages [ 690.417894] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 699.939304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 704.905972] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 707.905451] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 714.772567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 721.504385] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 731.091850] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 731.324353] Lustre: lustre-OST0001: new disk, initializing [ 731.327241] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 731.339948] Lustre: 26010:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 732.796784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 732.803809] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 732.935392] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 738.961337] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 750.105272] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 753.962465] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 769.006518] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 769.008926] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 769.020505] Lustre: Skipped 2 previous similar messages [ 769.023785] Lustre: Skipped 1 previous similar message [ 773.929274] Lustre: server umount lustre-MDT0000 complete [ 774.120289] LustreError: 23696:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 774.136493] LustreError: 23696:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 776.994063] LustreError: 23678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442423 with bad export cookie 12780075451929802539 [ 777.010024] LustreError: 23678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 777.013599] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 777.537042] Lustre: server umount lustre-MDT0001 complete [ 791.341514] Lustre: server umount lustre-OST0000 complete [ 795.105742] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788442426/real 1788442426] req@ffff99bfb1bdc000 x1875317110327168/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788442442 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 795.124491] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 795.137470] Lustre: Skipped 1 previous similar message [ 795.613987] Lustre: server umount lustre-OST0001 complete [ 815.331821] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 827.117604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 827.656484] LustreError: 29031:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 827.763234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 827.779737] Lustre: Skipped 3 previous similar messages [ 833.151370] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 842.145056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 847.316050] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 850.192942] Lustre: 30180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 856.373641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 862.615447] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 870.734810] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 876.978950] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 884.177190] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 888.124174] Lustre: 32056:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 893.421308] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 893.440450] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 893.445098] Lustre: Skipped 2 previous similar messages [ 893.460623] Lustre: Skipped 3 previous similar messages [ 899.719811] Lustre: server umount lustre-MDT0000 complete [ 903.392431] LustreError: 29013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442550 with bad export cookie 12780075451929803827 [ 903.398786] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 903.401382] LustreError: 29013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 903.651946] LustreError: 29027:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 903.665981] LustreError: 29027:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 903.754350] Lustre: server umount lustre-MDT0001 complete [ 909.263529] Lustre: server umount lustre-OST0000 complete [ 912.727732] Lustre: server umount lustre-OST0001 complete [ 921.629766] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:36:06 (1788442566) [ 950.972709] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 961.259996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 965.875220] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 975.192918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 980.512778] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 983.638687] Lustre: 36007:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 990.580699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 997.814958] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1006.647761] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1013.964477] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1022.211733] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1025.838624] Lustre: 37889:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1032.165901] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1032.175510] LustreError: Skipped 1 previous similar message [ 1032.179429] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1032.207586] Lustre: Skipped 3 previous similar messages [ 1032.215436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1032.222123] Lustre: Skipped 6 previous similar messages [ 1038.121057] Lustre: server umount lustre-MDT0000 complete [ 1042.203569] LustreError: 38041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442689 with bad export cookie 12780075451929804940 [ 1042.207865] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1042.216070] LustreError: 38041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1042.495198] Lustre: server umount lustre-MDT0001 complete [ 1057.301218] Lustre: server umount lustre-OST0000 complete [ 1058.272218] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788442689/real 1788442689] req@ffff99bf82f28000 x1875317110422400/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788442705 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1061.977612] Lustre: server umount lustre-OST0001 complete [ 1071.279391] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:38:36 (1788442716) [ 1088.149384] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1098.336599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1098.595797] LustreError: 40030:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1098.609819] LustreError: 40030:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 1098.699780] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1098.702788] Lustre: Skipped 7 previous similar messages [ 1103.040462] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1111.894985] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1117.319355] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1120.745150] Lustre: 41180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1127.698913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1135.218359] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1143.831336] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1150.475803] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1158.419620] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1162.265596] Lustre: 43058:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1163.101355] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1163.108496] Lustre: Skipped 1 previous similar message [ 1168.874807] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1168.884263] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1168.906188] Lustre: Skipped 4 previous similar messages [ 1172.725713] Lustre: server umount lustre-MDT0000 complete [ 1176.406171] LustreError: 41184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442823 with bad export cookie 12780075451929806046 [ 1176.412379] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1176.427293] LustreError: 41184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1182.809912] Lustre: server umount lustre-MDT0001 complete [ 1191.586447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1201.635993] LustreError: 3643:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff99bfab247480 x1875317110474112/t0(0) o250->MGC192.168.202.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1206.937860] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1212.412546] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1215.244565] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1220.812748] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1221.111509] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1221.120923] Lustre: Skipped 1 previous similar message [ 1228.770851] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1230.626866] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1237.647712] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1239.178466] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1246.707037] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1250.979426] Lustre: server umount lustre-MDT0000 complete [ 1254.769371] LustreError: 40010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442901 with bad export cookie 12780075451929807145 [ 1254.770405] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1254.776879] LustreError: 40010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1255.113793] Lustre: server umount lustre-MDT0001 complete [ 1269.535939] Lustre: server umount lustre-OST0000 complete [ 1272.096773] Lustre: 3645:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788442903/real 1788442903] req@ffff99bfb1b95880 x1875317110510464/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788442919 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1273.384722] Lustre: server umount lustre-OST0001 complete [ 1282.868884] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:42:07 (1788442927) [ 1299.629155] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1309.876410] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1314.906173] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1323.707361] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1329.493730] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1332.762762] Lustre: 48817:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1339.886757] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1347.158748] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1355.057664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1361.819140] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1368.453707] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1373.295736] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1390.560948] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1390.569828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1390.575099] Lustre: Skipped 22 previous similar messages [ 1394.873624] Lustre: server umount lustre-MDT0000 complete [ 1397.738981] LustreError: 47665:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1397.763027] LustreError: 47665:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 38 previous similar messages [ 1399.092053] LustreError: 48819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443046 with bad export cookie 12780075451929807957 [ 1399.095599] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1399.103610] LustreError: 48819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1399.470789] Lustre: server umount lustre-MDT0001 complete [ 1413.821241] Lustre: server umount lustre-OST0000 complete [ 1427.971647] Lustre: server umount lustre-OST0001 complete [ 1438.546847] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:44:43 (1788443083) [ 1441.604891] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1443.669931] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:44:48 (1788443088) [ 1454.971892] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 1462.662473] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1486.735297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1487.103502] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1487.234286] Lustre: lustre-MDT0000: new disk, initializing [ 1487.375076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1492.952763] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1505.393322] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1512.201786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1512.390896] Lustre: lustre-OST0000: new disk, initializing [ 1512.393714] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1512.396099] Lustre: Skipped 1 previous similar message [ 1512.399166] Lustre: 54891:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1514.319646] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1514.327122] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1514.361181] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1519.643542] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1529.419738] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1531.392902] Lustre: DEBUG MARKER: create test files [ 1570.796546] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1570.801660] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1570.815927] Lustre: Skipped 14 previous similar messages [ 1572.830875] Lustre: server umount lustre-OST0000 complete [ 1578.321058] Lustre: server umount lustre-MDT0000 complete [ 1591.662202] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 1595.050674] Key type lgssc unregistered [ 1595.340877] LNet: 56528:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1595.352163] LNetError: 56528:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1596.395786] LNet: Removed LNI 192.168.202.106@tcp [ 1597.273162] Key type .llcrypt unregistered [ 1597.280573] Key type ._llcrypt unregistered [ 1626.902354] Key type ._llcrypt registered [ 1626.904716] Key type .llcrypt registered [ 1626.981869] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1627.995836] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1628.032303] alg: No test for adler32 (adler32-zlib) [ 1629.065816] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1629.258369] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 1630.872165] Key type lgssc registered [ 1631.987656] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1639.529381] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1639.557309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1640.987324] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1645.237077] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1653.560792] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1662.157316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1662.452047] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1665.518887] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:96 to 0x240000400:129) [ 1665.588617] Lustre: 59263:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1665.597582] Lustre: 59263:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1665.616465] Lustre: 59263:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1665.624836] Lustre: 59263:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1665.635432] Lustre: 59263:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1665.642608] Lustre: 59263:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1670.267160] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1678.573418] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1683.979592] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1689.251092] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1696.230857] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1696.232110] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1696.275607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1696.739164] Lustre: server umount lustre-OST0000 complete [ 1700.922431] Lustre: server umount lustre-MDT0000 complete [ 1712.407968] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 1715.068401] Key type lgssc unregistered [ 1715.451077] LNet: 60876:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1715.467055] LNetError: 60876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1715.488149] LNet: Removed LNI 192.168.202.106@tcp [ 1716.489692] Key type .llcrypt unregistered [ 1716.495762] Key type ._llcrypt unregistered [ 1751.611450] Key type ._llcrypt registered [ 1751.614575] Key type .llcrypt registered [ 1751.791108] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1753.752614] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1753.795899] alg: No test for adler32 (adler32-zlib) [ 1755.006409] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1755.436507] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 1757.168217] Key type lgssc registered [ 1758.765655] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1767.882106] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1767.920205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1769.470210] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1775.532747] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1784.606512] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1792.412392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1792.810552] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1794.906284] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1795.048504] Lustre: 63711:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1795.053742] Lustre: 63711:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1795.074571] Lustre: 63711:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1795.082634] Lustre: 63711:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1795.101884] Lustre: 63711:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1795.113852] Lustre: 63711:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1800.472824] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1808.795995] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1814.310117] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1825.762119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1825.763090] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1825.775310] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1828.728151] Lustre: server umount lustre-OST0000 complete [ 1833.238173] Lustre: server umount lustre-MDT0000 complete [ 1846.702288] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 1850.126486] Key type lgssc unregistered [ 1850.332807] LNet: 65274:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1850.337193] LNetError: 65274:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1851.367872] LNet: Removed LNI 192.168.202.106@tcp [ 1852.345100] Key type .llcrypt unregistered [ 1852.346431] Key type ._llcrypt unregistered [ 1871.437501] Key type ._llcrypt registered [ 1871.438640] Key type .llcrypt registered [ 1871.486624] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 1883.203117] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1884.272823] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1884.405327] alg: No test for adler32 (adler32-zlib) [ 1885.440115] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1885.682252] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 1887.395728] Key type lgssc registered [ 1888.403822] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1922.533429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1922.623487] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1924.070217] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1924.133724] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1924.236617] Lustre: lustre-MDT0000: new disk, initializing [ 1924.375492] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1924.435762] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1930.643471] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1943.585288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1943.787762] Lustre: 68410:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1943.868405] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1943.873079] Lustre: Skipped 1 previous similar message [ 1943.980661] Lustre: lustre-MDT0001: new disk, initializing [ 1944.087029] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1944.131294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1944.155786] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1949.045804] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1959.121836] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1966.083473] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1971.993367] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1972.250913] Lustre: lustre-OST0000: new disk, initializing [ 1972.274500] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1972.285122] Lustre: 70381:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1972.346819] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1978.991243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1979.012407] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1979.251973] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1980.187852] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1991.752426] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1997.383936] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1997.636920] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2002.075479] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2002.308904] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2004.967682] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2004.969830] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2004.987755] Lustre: Skipped 1 previous similar message [ 2004.995906] Lustre: Skipped 1 previous similar message [ 2010.089280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2010.096733] Lustre: Skipped 1 previous similar message [ 2015.201821] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2015.214091] Lustre: Skipped 1 previous similar message [ 2018.272244] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2018.465370] Lustre: server umount lustre-OST0000 complete [ 2025.442202] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2025.446044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2025.453589] Lustre: Skipped 1 previous similar message [ 2025.458532] Lustre: Skipped 1 previous similar message [ 2028.623619] Lustre: server umount lustre-MDT0000 complete [ 2031.760520] LustreError: 68400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443678 with bad export cookie 3879756633892689773 [ 2031.764267] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2031.772423] LustreError: 68400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2032.110909] Lustre: server umount lustre-MDT0001 complete [ 2040.084430] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:54:44 (1788443684) [ 2050.727792] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2092.908641] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 2106.608752] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2149.397890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2149.786822] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2149.828381] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2149.938416] Lustre: lustre-MDT0000: new disk, initializing [ 2150.046707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2150.064715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2154.153441] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2164.072537] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2164.154984] Lustre: 76410:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2164.180936] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2164.183057] Lustre: Skipped 1 previous similar message [ 2164.234818] Lustre: lustre-MDT0001: new disk, initializing [ 2164.287493] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2164.307341] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2164.314618] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2169.297575] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2180.606761] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2187.298946] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2194.553955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2194.752588] Lustre: lustre-OST0000: new disk, initializing [ 2194.756125] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2194.762936] Lustre: 78379:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2194.848557] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2201.654814] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2203.249044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2203.271054] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2203.410134] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2212.738907] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2218.172266] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2218.434135] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2223.556297] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2223.758160] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2229.218037] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2229.219757] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2229.238985] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2229.240650] Lustre: Skipped 1 previous similar message [ 2229.255919] Lustre: Skipped 1 previous similar message [ 2232.053081] Lustre: server umount lustre-OST0000 complete [ 2235.873409] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2235.880439] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2239.458281] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2239.462540] Lustre: Skipped 1 previous similar message [ 2250.208126] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2250.529265] Lustre: server umount lustre-MDT0000 complete [ 2254.138537] LustreError: 77205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443901 with bad export cookie 3879756633892690648 [ 2254.141657] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2254.145095] LustreError: 77205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2254.355909] Lustre: server umount lustre-MDT0001 complete [ 2263.184658] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:58:28 (1788443908) [ 2269.917148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2270.336551] LustreError: 80794:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2270.356169] LustreError: 80794:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2270.390046] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2275.815093] LustreError: 80795:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2276.067343] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2280.932763] LustreError: 80794:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2286.066506] LustreError: 80795:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2288.376429] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2294.497425] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2302.628969] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2308.887492] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2316.338430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2316.849184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2316.853260] Lustre: Skipped 1 previous similar message [ 2323.527696] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2331.996077] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2336.453916] Lustre: 83469:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting c0f05759-9739-429f-a944-8a02c6400b1c at adminstrative request [ 2338.279565] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2338.293417] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2348.459729] Lustre: MGS: Client c0f05759-9739-429f-a944-8a02c6400b1c (at 0@lo) reconnecting [ 2348.472632] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x35d7aa1a4a239e4a to 0x35d7aa1a4a23a0ea [ 2348.508402] Lustre: MGC192.168.202.106@tcp: Connection restored to 0@lo (at 0@lo) [ 2373.510312] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2373.672849] Lustre: lustre-OST0001: new disk, initializing [ 2373.679212] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2373.683971] Lustre: 83861:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2380.697939] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2381.919700] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2381.944372] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2382.010262] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2390.044896] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2397.665616] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2397.667624] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2397.673245] LustreError: Skipped 1 previous similar message [ 2397.676332] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2397.714493] Lustre: Skipped 2 previous similar messages [ 2397.745971] Lustre: Skipped 5 previous similar messages [ 2406.882356] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2407.076807] Lustre: server umount lustre-OST0001 complete [ 2407.910056] LustreError: 83314:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2407.936041] LustreError: 83314:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2412.012201] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2412.042762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2417.940476] Lustre: server umount lustre-OST0000 complete [ 2429.143276] Lustre: server umount lustre-MDT0000 complete [ 2432.701437] LustreError: 81533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444079 with bad export cookie 3879756633892692202 [ 2432.705211] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2432.712200] LustreError: 81533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2433.043446] Lustre: server umount lustre-MDT0001 complete [ 2442.744439] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:01:26 (1788444086) [ 2444.881390] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2447.055100] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:01:31 (1788444091) [ 2448.888668] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2451.128788] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:01:35 (1788444095) [ 2453.731849] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2455.655949] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:01:40 (1788444100) [ 2457.934915] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2461.218891] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:01:45 (1788444105) [ 2463.343205] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2469.211843] loop: module loaded [ 2469.256833] loop0: detected capacity change from 0 to 4294967296 [ 2469.336380] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2470.819777] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2475.663704] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2480.277482] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2507.870371] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2507.971042] LustreError: 87997:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2507.983270] LustreError: 87997:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2507.992313] LustreError: 87997:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2507.999079] LustreError: 87997:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2526.721502] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 2537.245314] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2576.235506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2576.697207] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2576.738885] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2576.790289] Lustre: lustre-MDT0000: new disk, initializing [ 2576.905895] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2576.910982] Lustre: Skipped 1 previous similar message [ 2576.942784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2581.792261] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2594.394321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2594.539852] Lustre: 90864:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2594.907250] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2601.007286] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2610.292464] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2616.362455] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2623.210260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2623.697562] Lustre: 92833:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2630.254261] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2632.307728] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2632.457268] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2640.302936] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2645.912555] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2646.102308] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2650.603339] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2650.849267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2653.154976] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2653.159109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2653.173222] Lustre: Skipped 3 previous similar messages [ 2653.185105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2653.197745] Lustre: Skipped 10 previous similar messages [ 2658.541805] Lustre: server umount lustre-OST0000 complete [ 2667.730782] Lustre: server umount lustre-MDT0000 complete [ 2668.513512] LustreError: 90871:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2668.530395] LustreError: 90871:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2671.011103] LustreError: 90855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444317 with bad export cookie 3879756633892692825 [ 2671.018360] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2671.028651] LustreError: 90855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2671.797887] Lustre: server umount lustre-MDT0001 complete [ 2679.742341] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:05:24 (1788444324) [ 2686.987719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2687.533587] LustreError: 95245:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2692.163731] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2701.919739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2707.112931] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2714.998570] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2721.505939] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2730.545988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2730.900131] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2730.905977] Lustre: Skipped 4 previous similar messages [ 2737.718529] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2746.934584] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2752.746919] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2752.996604] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2758.635307] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2758.910285] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2766.212504] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2766.414749] Lustre: lustre-OST0001: new disk, initializing [ 2766.417971] Lustre: Skipped 2 previous similar messages [ 2766.422912] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2766.429223] Lustre: Skipped 3 previous similar messages [ 2766.436278] Lustre: 98614:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2773.665099] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2773.666198] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2773.678519] Lustre: Skipped 2 previous similar messages [ 2773.679543] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2773.777899] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2782.403951] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2787.072226] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2787.286660] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2792.395609] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2792.641746] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2812.428146] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2839.026561] Lustre: Failing over lustre-MDT0000 [ 2840.548300] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2840.563609] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2840.567106] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2840.592480] Lustre: Skipped 6 previous similar messages [ 2840.598083] Lustre: Skipped 6 previous similar messages [ 2841.881461] Lustre: server umount lustre-MDT0000 complete [ 2841.889043] LustreError: 95242:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2841.922569] LustreError: 95242:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2862.048351] Lustre: 66629:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788444492/real 1788444492] req@00000000ccb763c5 x1875318965294208/t0(0) o400->MGC192.168.202.106@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788444508 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2862.068466] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2867.659159] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2867.665090] LDISKFS-fs (dm-0): recovery complete [ 2867.674737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2872.312284] LustreError: 66628:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000007ca4ef05 x1875318965302784/t0(0) o250->MGC192.168.202.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2872.643345] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2873.603606] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2873.606939] Lustre: 95241:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2877.604076] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2877.935251] Lustre: 95973:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2877.948968] Lustre: 95973:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2877.960354] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2878.044625] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2878.368096] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2878.704993] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2878.716735] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2879.033076] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2879.042787] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2879.057475] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2879.070675] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2879.737103] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2879.759375] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2880.080128] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2880.089657] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2881.372407] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2881.398242] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2882.069718] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2882.082260] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2882.408154] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2882.413283] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2885.585774] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2885.591759] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2886.251873] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2886.264152] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2886.584329] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2886.612340] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2893.594606] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2893.609127] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2894.254305] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2894.266605] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2894.601111] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2894.611700] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2909.827106] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2909.849396] Lustre: 101467:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 2910.508189] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2910.523875] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2910.848108] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2910.853764] LustreError: 101467:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2933.080275] Lustre: 101467:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2933.102391] Lustre: 101467:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2933.111523] Lustre: 101467:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client fc8c1509-32a4-4493-9126-127051e9ec6c@192.168.202.6@tcp [ 2933.129789] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2933.136726] Lustre: 101467:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000deb2b60e x1875318952496512/t0(8589934769) o101->fc8c1509-32a4-4493-9126-127051e9ec6c@192.168.202.6@tcp:590/0 lens 592/0 e 0 to 0 dl 1788444590 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2933.181585] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2933.192735] Lustre: lustre-MDT0000: Denying connection for new client fc8c1509-32a4-4493-9126-127051e9ec6c (at 192.168.202.6@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2933.224801] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2933.276736] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2933.281129] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2933.283632] Lustre: Skipped 2 previous similar messages [ 2933.340970] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2933.342946] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2954.723148] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2954.733765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2959.317522] Lustre: server umount lustre-OST0000 complete [ 2962.929214] LustreError: 97065:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2962.964855] LustreError: 97065:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 41 previous similar messages [ 2973.159672] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2973.170437] Lustre: Skipped 8 previous similar messages [ 2978.273041] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2978.691568] Lustre: server umount lustre-OST0001 complete [ 2988.988697] Lustre: server umount lustre-MDT0000 complete [ 2992.904828] LustreError: 95225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444639 with bad export cookie 3879756633892744044 [ 2992.906908] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2992.916294] LustreError: 95225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2993.447596] Lustre: server umount lustre-MDT0001 complete [ 3002.995625] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:10:47 (1788444647) [ 3010.842587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3010.847231] Lustre: *** cfs_fail_loc=197, val=0*** [ 3026.400899] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3031.520954] LustreError: 103444:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.106@tcp: failed processing log, type 4: rc = -110 [ 3036.667880] Lustre: *** cfs_fail_loc=197, val=0*** [ 3036.675379] Lustre: Skipped 65 previous similar messages [ 3057.248285] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3057.259646] Lustre: Skipped 2 previous similar messages [ 3064.182379] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3071.583769] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3073.545779] Lustre: server umount lustre-OST0000 complete [ 3081.302450] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 10:12:06 (1788444726) [ 3101.430249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3101.748114] LustreError: 105068:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3101.760437] LustreError: 105068:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 3107.088115] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3119.614417] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3124.254302] LustreError: 105099:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3124.278123] LustreError: 105099:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3124.292356] LustreError: 105099:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 23, retries 0, failed: rc = -5 [ 3124.706271] Lustre: server umount lustre-MDT0000 complete [ 3161.880840] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 3173.768649] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3212.811835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3213.071520] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3213.091851] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3213.159854] Lustre: lustre-MDT0000: new disk, initializing [ 3213.270271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3217.944106] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3228.897560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3229.023272] Lustre: 109274:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3229.323858] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3233.860217] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3244.535331] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3251.219161] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3258.808828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3259.111634] Lustre: 111244:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3264.461170] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3265.333651] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3275.715428] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3279.965909] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3280.181205] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3284.922200] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3285.217406] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3291.109060] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3291.114027] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3291.125252] LustreError: Skipped 3 previous similar messages [ 3291.163292] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3291.165811] Lustre: Skipped 6 previous similar messages [ 3291.228301] Lustre: Skipped 7 previous similar messages [ 3302.368131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3302.689667] Lustre: server umount lustre-OST0000 complete [ 3314.321622] Lustre: server umount lustre-MDT0000 complete [ 3318.973314] LustreError: 112787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444965 with bad export cookie 3879756633892754614 [ 3318.974271] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3318.982681] LustreError: 112787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3319.466847] Lustre: server umount lustre-MDT0001 complete [ 3328.710574] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:16:13 (1788444973) [ 3338.972754] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3343.090461] Key type lgssc unregistered [ 3343.529823] LNet: 113739:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3343.558512] LNetError: 113739:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3343.619049] LNet: Removed LNI 192.168.202.106@tcp [ 3345.076442] Key type .llcrypt unregistered [ 3345.082434] Key type ._llcrypt unregistered [ 3368.241474] Key type ._llcrypt registered [ 3368.247781] Key type .llcrypt registered [ 3368.396541] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 3386.094043] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3387.139398] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3387.152406] alg: No test for adler32 (adler32-zlib) [ 3388.432670] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3388.797464] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3390.531051] Key type lgssc registered [ 3392.349373] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3435.148606] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3435.185252] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3436.500407] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3436.531873] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3436.622367] Lustre: lustre-MDT0000: new disk, initializing [ 3436.736418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3436.758489] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3441.672131] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3450.883267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3451.241307] Lustre: lustre-OST0000: new disk, initializing [ 3451.247153] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3451.250328] Lustre: Skipped 1 previous similar message [ 3451.254646] Lustre: 117914:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3451.389917] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3454.676168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3454.686275] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3454.739936] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3458.395385] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3469.720145] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3470.080271] Lustre: lustre-OST0001: new disk, initializing [ 3470.090095] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3470.102788] Lustre: 118776:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3470.206075] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3476.562655] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3476.623201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3476.629305] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3476.688413] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3487.201104] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3487.206316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3487.227278] Lustre: Skipped 1 previous similar message [ 3487.278965] Lustre: Skipped 1 previous similar message [ 3489.383956] Lustre: server umount lustre-MDT0000 complete [ 3497.452570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3504.216793] LustreError: 117097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445151 with bad export cookie 7429934652312588582 [ 3504.227698] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3504.229767] LustreError: 117097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3504.402793] Lustre: server umount lustre-OST0000 complete [ 3508.589402] Lustre: server umount lustre-OST0001 complete [ 3516.476475] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:19:21 (1788445161) [ 3539.274404] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:19:44 (1788445184) [ 3550.411411] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3573.488653] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:20:18 (1788445218) [ 3616.051772] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 3626.181559] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3664.929676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3665.191866] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3665.228734] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3665.356989] Lustre: lustre-MDT0000: new disk, initializing [ 3665.448728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3665.487875] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3671.156396] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3681.834372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3681.983172] Lustre: 126528:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3682.032983] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3682.036713] Lustre: Skipped 1 previous similar message [ 3682.100367] Lustre: lustre-MDT0001: new disk, initializing [ 3682.211282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3682.258470] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3682.276235] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3686.944447] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3696.480098] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3703.061620] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3710.625995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3710.989046] Lustre: lustre-OST0000: new disk, initializing [ 3710.992926] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3710.998972] Lustre: 128496:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3711.087486] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3716.781642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3716.802149] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3717.006866] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3719.749270] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3731.662285] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3736.612562] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3736.741288] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3741.809509] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3741.932657] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3747.813808] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3747.836177] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3747.849995] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3747.858909] Lustre: Skipped 1 previous similar message [ 3750.419784] Lustre: server umount lustre-OST0000 complete [ 3758.053457] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3758.059730] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3758.075529] Lustre: Skipped 2 previous similar messages [ 3758.095888] Lustre: Skipped 1 previous similar message [ 3760.645524] Lustre: server umount lustre-MDT0000 complete [ 3764.209139] LustreError: 126521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445411 with bad export cookie 7429934652312589807 [ 3764.211666] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3764.214158] LustreError: 126521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3764.509625] Lustre: server umount lustre-MDT0001 complete [ 3772.854391] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:23:38 (1788445418) [ 3779.575931] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3780.187653] LustreError: 130911:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3780.200792] LustreError: 130911:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3780.264606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3784.972433] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3785.698815] LustreError: 130912:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3790.838133] LustreError: 130911:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3794.305541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3801.314453] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3815.388375] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3822.719589] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3831.271043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3831.894235] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3831.908390] Lustre: Skipped 1 previous similar message [ 3840.185846] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3848.909723] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3858.542532] Lustre: *** cfs_fail_loc=159, val=0*** [ 3874.114442] Lustre: lustre-MDT0000: Client 13fb5330-00ea-4a3d-84b2-eebaa9bc4a53 (at 192.168.202.6@tcp) reconnecting [ 3876.634680] Lustre: *** cfs_fail_loc=159, val=0*** [ 3876.647533] Lustre: Skipped 5 previous similar messages [ 3893.107306] Lustre: lustre-MDT0000: Client 13fb5330-00ea-4a3d-84b2-eebaa9bc4a53 (at 192.168.202.6@tcp) reconnecting [ 3900.901326] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3900.920277] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3900.940488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3908.080857] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3908.097877] Lustre: Skipped 3 previous similar messages [ 3913.184421] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3913.372620] Lustre: server umount lustre-OST0000 complete [ 3917.796258] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3917.808772] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3917.821380] Lustre: Skipped 1 previous similar message [ 3917.831221] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3917.835742] Lustre: Skipped 2 previous similar messages [ 3924.036501] Lustre: server umount lustre-MDT0000 complete [ 3928.046609] LustreError: 130891:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445574 with bad export cookie 7429934652312590682 [ 3928.050867] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3928.054729] LustreError: 130891:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3928.352992] Lustre: server umount lustre-MDT0001 complete [ 3940.727764] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3944.043677] Key type lgssc unregistered [ 3944.337243] LNet: 134703:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3944.345510] LNetError: 134703:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3945.390677] LNet: Removed LNI 192.168.202.106@tcp [ 3946.142914] Key type .llcrypt unregistered [ 3946.147299] Key type ._llcrypt unregistered [ 3976.001900] Key type ._llcrypt registered [ 3976.006435] Key type .llcrypt registered [ 3976.204957] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3978.062513] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3978.103988] alg: No test for adler32 (adler32-zlib) [ 3979.178799] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3979.466543] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3981.192584] Key type lgssc registered [ 3982.471564] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3992.063809] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3992.093709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3993.724735] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3999.104721] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4003.309978] LustreError: 136118:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4003.330157] LustreError: 136118:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4008.422094] LustreError: 136119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4009.611325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4010.173796] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4015.821676] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4024.222690] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4031.601345] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4039.198548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4039.751735] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4046.777860] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4055.484117] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4063.363565] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4076.416242] Lustre: *** cfs_fail_loc=159, val=0*** [ 4091.698902] Lustre: lustre-MDT0000: Client baf8e0de-a31c-4a0a-8545-93edcb60791f (at 192.168.202.6@tcp) reconnecting [ 4094.573876] Lustre: *** cfs_fail_loc=159, val=0*** [ 4094.578198] Lustre: Skipped 1 previous similar message [ 4110.132533] Lustre: lustre-MDT0001: Client baf8e0de-a31c-4a0a-8545-93edcb60791f (at 192.168.202.6@tcp) reconnecting [ 4111.784440] Lustre: *** cfs_fail_loc=159, val=0*** [ 4111.786122] Lustre: Skipped 2 previous similar messages [ 4128.064744] Lustre: lustre-MDT0001: Client baf8e0de-a31c-4a0a-8545-93edcb60791f (at 192.168.202.6@tcp) reconnecting [ 4133.767429] Lustre: 139181:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4137.122690] Lustre: *** cfs_fail_loc=159, val=0*** [ 4137.124324] Lustre: Skipped 3 previous similar messages [ 4153.700619] Lustre: lustre-MDT0000: Client a6739f32-683c-496e-964b-cc45feeab0aa (at 192.168.202.6@tcp) reconnecting [ 4155.641548] Lustre: *** cfs_fail_loc=159, val=0*** [ 4155.647532] Lustre: Skipped 14 previous similar messages [ 4172.152953] Lustre: lustre-MDT0000: Client a6739f32-683c-496e-964b-cc45feeab0aa (at 192.168.202.6@tcp) reconnecting [ 4173.653232] Lustre: 139474:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4178.915742] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4178.932061] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4178.958130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4181.473403] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4181.476019] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4181.489181] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4181.518045] Lustre: Skipped 1 previous similar message [ 4186.600534] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4186.608332] Lustre: Skipped 1 previous similar message [ 4191.716144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4191.721315] Lustre: Skipped 1 previous similar message [ 4192.224305] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4192.464179] Lustre: server umount lustre-OST0000 complete [ 4196.841791] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4196.861658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4202.581885] Lustre: server umount lustre-MDT0000 complete [ 4206.451659] LustreError: 136100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445853 with bad export cookie 15935046870642662764 [ 4206.455073] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4206.459649] LustreError: 136100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4206.911768] Lustre: server umount lustre-MDT0001 complete [ 4218.012323] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 4220.584747] Key type lgssc unregistered [ 4220.912808] LNet: 140530:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4220.922357] LNetError: 140530:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4220.939457] LNet: Removed LNI 192.168.202.106@tcp [ 4221.951676] Key type .llcrypt unregistered [ 4221.954627] Key type ._llcrypt unregistered [ 4247.913177] Key type ._llcrypt registered [ 4247.914427] Key type .llcrypt registered [ 4248.059895] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 4249.504777] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4249.569505] alg: No test for adler32 (adler32-zlib) [ 4250.728610] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4251.072331] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 4252.776331] Key type lgssc registered [ 4253.827483] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4261.795484] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4261.825581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4263.335975] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4267.093609] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4273.664804] LustreError: 141962:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4273.691551] LustreError: 141962:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4275.983443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4276.255863] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4281.802673] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4288.373455] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4294.003560] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4300.054046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4300.406962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4301.458559] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4306.844553] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4308.606203] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4313.874302] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4319.911818] Lustre: 144542:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4322.717227] Lustre: 144591:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4324.242678] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4325.348860] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4328.991730] Lustre: 144738:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4329.825470] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4330.864113] Lustre: 144836:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4337.634393] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4337.644505] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4337.656772] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4339.681857] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4339.695851] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4339.701367] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4339.719924] Lustre: Skipped 1 previous similar message [ 4342.708956] Lustre: server umount lustre-OST0000 complete [ 4347.874196] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4347.889654] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4347.914036] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4349.924700] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4349.925929] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4354.253934] Lustre: server umount lustre-MDT0000 complete [ 4355.043589] LustreError: 142689:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4355.063211] LustreError: 142689:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4358.349412] LustreError: 141943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446005 with bad export cookie 1319009053439239289 [ 4358.351686] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4358.357765] LustreError: 141943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4358.771775] Lustre: server umount lustre-MDT0001 complete [ 4370.951747] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 4374.071934] Key type lgssc unregistered [ 4374.523569] LNet: 145901:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4374.544762] LNetError: 145901:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4375.597608] LNet: Removed LNI 192.168.202.106@tcp [ 4376.548737] Key type .llcrypt unregistered [ 4376.550748] Key type ._llcrypt unregistered [ 4400.670914] Key type ._llcrypt registered [ 4400.673445] Key type .llcrypt registered [ 4400.781674] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 4401.904757] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4401.997523] alg: No test for adler32 (adler32-zlib) [ 4403.226992] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4403.568389] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 4405.227315] Key type lgssc registered [ 4406.613653] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4415.322195] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4415.341245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4416.809945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4421.156883] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4426.210015] LustreError: 147317:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4426.239328] LustreError: 147317:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4429.370394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4429.872179] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4434.385765] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4442.967400] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4449.863452] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4459.057949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4459.481142] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4462.605711] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4467.270919] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4474.863923] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4475.468286] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4482.261259] Lustre: *** cfs_fail_loc=159, val=0*** [ 4497.703099] Lustre: lustre-MDT0000: Client b4944071-d4af-4383-b6a3-7c58d0cac7c4 (at 192.168.202.6@tcp) reconnecting [ 4505.570291] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4505.574289] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4505.577733] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4505.620764] Lustre: Skipped 1 previous similar message [ 4510.208824] Lustre: server umount lustre-OST0000 complete [ 4515.810086] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4515.833686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4515.850026] Lustre: Skipped 2 previous similar messages [ 4520.356948] Lustre: server umount lustre-MDT0000 complete [ 4520.931837] LustreError: 147317:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4520.958974] LustreError: 147317:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4524.656985] LustreError: 147298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446171 with bad export cookie 14122535796664124731 [ 4524.665104] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4524.670464] LustreError: 147298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4525.109408] Lustre: server umount lustre-MDT0001 complete [ 4536.495452] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 4539.374498] Key type lgssc unregistered [ 4539.705969] LNet: 150985:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4539.720874] LNetError: 150985:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4539.752872] LNet: Removed LNI 192.168.202.106@tcp [ 4540.742887] Key type .llcrypt unregistered [ 4540.745531] Key type ._llcrypt unregistered [ 4567.581369] Key type ._llcrypt registered [ 4567.584384] Key type .llcrypt registered [ 4567.690216] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 4568.809379] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4568.908388] alg: No test for adler32 (adler32-zlib) [ 4569.986429] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4570.208771] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 4571.864182] Key type lgssc registered [ 4573.071600] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4581.014316] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4581.035534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4582.561790] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4586.981525] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4592.172421] LustreError: 152401:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4592.221108] LustreError: 152401:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4595.810511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4596.367921] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4601.090224] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4609.608861] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4615.525381] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4622.759649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4623.049788] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4627.191474] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4629.267829] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4637.684780] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4641.526059] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4662.312928] Lustre: lustre-MDT0001: evicting e9519062-4e71-4b9a-b039-2a5855b93129 (at 192.168.202.6@tcp) by administrative request [ 4662.321633] Lustre: Skipped 1 previous similar message [ 4692.961335] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4692.964818] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4692.972056] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4693.999132] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4694.008838] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4694.023809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4696.039427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4696.059539] Lustre: Skipped 1 previous similar message [ 4697.418877] Lustre: server umount lustre-OST0000 complete [ 4703.714117] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4703.723087] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4703.744858] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4706.275121] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4708.032486] Lustre: server umount lustre-MDT0000 complete [ 4711.394651] LustreError: 152396:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4711.417445] LustreError: 152396:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4713.089575] LustreError: 152380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446360 with bad export cookie 15632114542563363522 [ 4713.098615] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4713.109918] LustreError: 152380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4713.397610] Lustre: server umount lustre-MDT0001 complete [ 4725.993213] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 4729.397590] Key type lgssc unregistered [ 4729.820836] LNet: 156474:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4729.841764] LNetError: 156474:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4729.860798] LNet: Removed LNI 192.168.202.106@tcp [ 4731.491472] Key type .llcrypt unregistered [ 4731.508730] Key type ._llcrypt unregistered [ 4811.721983] Key type ._llcrypt registered [ 4811.724292] Key type .llcrypt registered [ 4811.824285] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 4813.557961] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4813.616280] alg: No test for adler32 (adler32-zlib) [ 4815.160854] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4815.638792] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 4817.456166] Key type lgssc registered [ 4818.434975] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4826.844409] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4826.886180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4828.495700] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4834.363723] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4838.883608] LustreError: 158977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4838.898268] LustreError: 158977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4842.541693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4842.759151] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4847.506473] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4855.786739] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4862.157816] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4869.698899] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4870.146945] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4873.222877] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4877.421911] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 4885.384467] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4887.547045] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4897.768419] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4897.773683] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4897.792835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4897.807380] Lustre: Skipped 1 previous similar message [ 4897.837135] Lustre: Skipped 1 previous similar message [ 4901.673913] Lustre: server umount lustre-OST0000 complete [ 4908.004709] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4908.006392] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4908.010944] Lustre: Skipped 1 previous similar message [ 4908.033744] Lustre: Skipped 1 previous similar message [ 4912.460768] Lustre: server umount lustre-MDT0000 complete [ 4913.125233] LustreError: 158974:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4913.151570] LustreError: 158974:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4916.822804] LustreError: 158957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446563 with bad export cookie 10093906185149304370 [ 4916.829755] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4916.837163] LustreError: 158957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4917.223435] Lustre: server umount lustre-MDT0001 complete [ 4929.400915] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 4932.497876] Key type lgssc unregistered [ 4932.797669] LNet: 162522:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4932.806511] LNetError: 162522:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4932.826167] LNet: Removed LNI 192.168.202.106@tcp [ 4933.966243] Key type .llcrypt unregistered [ 4933.973324] Key type ._llcrypt unregistered [ 4960.857126] Key type ._llcrypt registered [ 4960.868057] Key type .llcrypt registered [ 4961.067490] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 4962.450236] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4962.509503] alg: No test for adler32 (adler32-zlib) [ 4963.998373] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4964.617649] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 4966.408197] Key type lgssc registered [ 4967.701833] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4984.021990] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:43:48 (1788446628) [ 5000.145737] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 5010.402733] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5050.945133] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5050.964400] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5052.285046] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5052.336422] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5052.449885] Lustre: lustre-MDT0000: new disk, initializing [ 5052.575863] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5052.604583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5057.213681] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5068.717831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5068.867601] Lustre: 166787:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5068.933349] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5068.937905] Lustre: Skipped 1 previous similar message [ 5069.044341] Lustre: lustre-MDT0001: new disk, initializing [ 5069.190060] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5069.268859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5069.277358] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5074.409244] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5085.215973] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5091.544861] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5098.832053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5099.124598] Lustre: lustre-OST0000: new disk, initializing [ 5099.128492] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5099.135433] Lustre: 168757:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5099.240983] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5106.117277] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5108.262532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5108.275994] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5108.356306] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5115.590572] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5124.067632] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5124.075186] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5124.077540] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5124.087991] Lustre: Skipped 1 previous similar message [ 5124.124055] Lustre: Skipped 1 previous similar message [ 5129.190027] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5129.198376] Lustre: Skipped 1 previous similar message [ 5130.150669] Lustre: server umount lustre-OST0000 complete [ 5134.305928] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5134.312513] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5134.317519] Lustre: Skipped 1 previous similar message [ 5139.426918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5139.433283] Lustre: Skipped 2 previous similar messages [ 5144.549149] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5144.560644] Lustre: Skipped 1 previous similar message [ 5148.128252] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5148.371937] Lustre: server umount lustre-MDT0000 complete [ 5152.776986] LustreError: 166780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446799 with bad export cookie 7242968854830748368 [ 5152.796087] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5152.797123] LustreError: 166780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5153.349247] Lustre: server umount lustre-MDT0001 complete [ 5165.162715] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 5168.178609] Key type lgssc unregistered [ 5168.502706] LNet: 170666:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5168.514634] LNetError: 170666:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5168.531851] LNet: Removed LNI 192.168.202.106@tcp [ 5169.645535] Key type .llcrypt unregistered [ 5169.647446] Key type ._llcrypt unregistered [ 5199.172874] Key type ._llcrypt registered [ 5199.176984] Key type .llcrypt registered [ 5199.389918] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5201.437689] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5201.481334] alg: No test for adler32 (adler32-zlib) [ 5202.763799] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 5203.144977] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 5204.856204] Key type lgssc registered [ 5206.230572] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5214.735721] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5214.756136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5216.218699] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5221.159199] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5226.471866] LustreError: 172091:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5226.490860] LustreError: 172091:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5229.791980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5230.219277] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5234.967411] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5242.150763] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5246.974282] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5253.863692] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5254.173853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5260.836759] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5261.376274] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5269.086832] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5274.805152] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5274.829661] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5274.851798] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5274.863234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.883186] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5275.719957] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5275.731752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5275.739791] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5275.746270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.753310] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5276.660637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.514769] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5277.518616] Lustre: Skipped 1 previous similar message [ 5277.529236] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5277.536154] Lustre: Skipped 1 previous similar message [ 5277.538994] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5277.542536] Lustre: Skipped 1 previous similar message [ 5277.545274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.555144] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5277.562055] Lustre: Skipped 1 previous similar message [ 5278.345746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.116543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.932377] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5279.937365] Lustre: Skipped 2 previous similar messages [ 5279.952109] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5279.961121] Lustre: Skipped 2 previous similar messages [ 5279.970665] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5279.975547] Lustre: Skipped 2 previous similar messages [ 5279.979461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.998431] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5280.006729] Lustre: Skipped 2 previous similar messages [ 5280.780286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.654354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.376327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.247353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.062925] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5284.068856] Lustre: Skipped 4 previous similar messages [ 5284.084258] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5284.100216] Lustre: Skipped 4 previous similar messages [ 5284.103907] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5284.108555] Lustre: Skipped 4 previous similar messages [ 5284.112192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.131135] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5284.133700] Lustre: Skipped 4 previous similar messages [ 5285.186303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.408467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.422640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.387222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.334699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.316082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.399482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.372811] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5292.380291] Lustre: Skipped 7 previous similar messages [ 5292.413081] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5292.427444] Lustre: Skipped 7 previous similar messages [ 5292.433939] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5292.444826] Lustre: Skipped 7 previous similar messages [ 5292.449901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.463371] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5292.468225] Lustre: Skipped 7 previous similar messages [ 5293.480611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.309486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.324566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.340959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.292607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.112454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.009955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.002361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.797316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.596380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.408827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.338045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.292976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.065324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.961614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.931251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.823431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.723770] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5308.734320] Lustre: Skipped 17 previous similar messages [ 5308.751210] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5308.760093] Lustre: Skipped 17 previous similar messages [ 5308.763919] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5308.775422] Lustre: Skipped 17 previous similar messages [ 5308.781052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.794402] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5308.802716] Lustre: Skipped 17 previous similar messages [ 5309.738230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.865838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.697481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.674630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.536091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.384778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.340910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.308832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.233750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.190630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.187564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.335101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.484033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.449982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.450860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.345927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.226938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.185405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.011555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.871395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.825385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.774241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.753128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.610513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.587731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.554638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.566923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.455755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.347356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.310451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.253857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.040062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.034084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.755944] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5340.763416] Lustre: Skipped 33 previous similar messages [ 5340.773121] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5340.779953] Lustre: Skipped 33 previous similar messages [ 5340.782975] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5340.787682] Lustre: Skipped 33 previous similar messages [ 5340.791279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.800023] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5340.806518] Lustre: Skipped 33 previous similar messages [ 5341.650367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.445796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.427951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.249109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.180500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.102224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.871927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.723068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.669262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.539731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.388860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.080897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.867914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.619462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.346542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.112674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.962431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.689744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.573393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.434703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.491924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.428979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.492385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.372879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.310184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.180732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.229912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.412852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.595538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.553587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.629563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.026484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.871902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.842715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.917983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.857564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.748599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.821413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.009878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.967475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.683064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.494860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.303838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.335651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.812641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.741573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.749852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.046509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.023559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.904271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.661370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.549733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.450342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.468240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.560607] LustreError: 172110:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5392.564808] LustreError: 172110:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5392.581602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.674362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.599979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.882408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.731769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.558507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.586152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.574988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.540279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.598457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.635641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.513781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.449931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.220536] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5405.223704] Lustre: Skipped 67 previous similar messages [ 5405.242482] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5405.251369] Lustre: Skipped 67 previous similar messages [ 5405.255611] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5405.257716] Lustre: Skipped 67 previous similar messages [ 5405.260995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.267983] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5405.282084] Lustre: Skipped 67 previous similar messages [ 5406.190589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.143664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.069559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.251347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.119540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.969842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.889761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.894538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.834552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.875410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.880180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.806094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.842606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.854208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.953383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.069245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.023811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.858224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.725055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.598882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.460467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.393593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.298709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.252482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.198028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.242279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.351724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.355202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.266485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.258985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.593825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.534820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.471600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.480167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.309766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.184392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.985965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.808477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.607501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.433054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.357139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.543162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.560503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.827335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.664799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.439787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.110170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.147404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.079771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.298058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.173384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.912565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.796281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.823322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.868563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.594798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.611205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.574819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.420954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.648468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.075642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.351940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.602488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.884659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.337462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.262449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.187801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.301573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.131158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.945946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.945972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.061258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.748146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.730260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.694703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.973976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.928062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.730529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.390522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.232534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.192873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.144849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.015562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.975684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.780082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.835665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.736776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.593739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.409444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.471208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.476026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.401469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.017535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.840654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.764342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.811907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.806525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.923806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.109177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.436464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.398928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.206417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.872495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.816347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.664771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.314710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.129903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.887488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.704623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.470632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.276135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.126658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.848622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.734711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.704177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.554971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.452365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.318564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.196594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.039584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.862563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.757299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.427646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.307477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.111923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.794957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.562550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.214322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.946486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.869962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.626799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.570724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.479482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.491964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.327806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.298890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.960232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.533199] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5533.540446] Lustre: Skipped 137 previous similar messages [ 5533.560628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5533.571569] Lustre: Skipped 137 previous similar messages [ 5533.574620] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5533.578163] Lustre: Skipped 137 previous similar messages [ 5533.581609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.587988] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5533.590457] Lustre: Skipped 137 previous similar messages [ 5534.424137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.448906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.357647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.191461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.251377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.556436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.368850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.321386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.667293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.378867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.104968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.346725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.182834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.868912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.802130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.836488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.813701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.109809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.295299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.082409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.340604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.357789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.442810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.403907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.438796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.426255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.661121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.458196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.439462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.474356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.507820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.922464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.532052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.823601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.522659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.250176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.094732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.257781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.024906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.933504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.842177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.965588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.949817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.233760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.404396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.423635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.487569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.610120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.723414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.801720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.833440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.873683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.743069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.707875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.599676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.627337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.576740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.685093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.796476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.754235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.887571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.867135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.978537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.052253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.947300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.839897] LustreError: 172110:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5600.848129] LustreError: 172110:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5600.857931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.853051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.686264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.588249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.429355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.567804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.540701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.636921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.776547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.031224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.956729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.250258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.157909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.388071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.451880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.177320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.231698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.297005] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.294803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.003203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.817074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.599828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.331873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.106899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.849199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.735853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.550896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.288397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.120909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.116577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.942362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.948555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.694111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.622369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.531329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.371584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.114862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.056472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.910558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.770634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.844835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.867611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.774541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.571028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.590220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.468408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.479322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.279597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.095481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.794228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.623848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.359851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.260765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.075181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.903146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.825218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.887962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.807722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.703659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.496523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.513759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.325910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.144081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.940696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.641850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.473642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.249436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.240738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.022944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.941959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.635530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.406639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.211719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.991209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.769816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.740461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.539896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.468680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.444143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.451226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.357378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.170859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.947446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.841405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.776895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.738854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.528743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.395018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.179247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.027763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.991745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.829670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.742641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.812796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.619537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.441619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.125828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.050729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.762606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.634148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.367443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.337081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.162467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.135249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.089880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.086882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.037631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.080131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.899826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.545624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.467741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.634617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.542424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.381065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.263745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.295441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.074426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.825875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.432705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.181804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.210445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.157253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.270906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.534402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.777724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.219891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.548075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.424961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.718144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.822600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.780880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.805972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.633812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.029259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.228756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.237754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.905732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.988371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.945841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.148604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.110482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.041550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.051208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.150238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.419258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.276643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.685796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.644172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.726744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.494046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.382631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.092471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.763244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.828186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.039248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.292098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.265166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.695771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.022747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.517603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.386966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.581555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.487184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.478460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.411619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.282233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.045813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.125140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.872742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.075274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.833927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.970931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.932489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.204346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.983527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.962058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.779971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.762873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.857588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.936301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.016057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.117509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.318039] LustreError: 172110:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5772.331966] LustreError: 172110:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5772.349939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.592299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.908852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.090734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.034695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.003638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.896339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.840633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.852421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.278793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.378772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.175754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.114616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.879018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.910386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.018809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.186569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.113763] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5790.117560] Lustre: Skipped 264 previous similar messages [ 5790.133279] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5790.138533] Lustre: Skipped 264 previous similar messages [ 5790.141221] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5790.143717] Lustre: Skipped 264 previous similar messages [ 5790.147370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.155847] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5790.162187] Lustre: Skipped 264 previous similar messages [ 5791.249571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.059850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.740839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.438448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.496846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.249491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.954983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.864179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.586397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.394400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.146898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.015511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.007808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.936161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.930227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.572721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.601060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.595876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.376141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.963324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.894656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.737861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.898751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.734965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.355388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.080904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.843614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.840385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.832425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.838419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.746955] LustreError: 172110:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5816.756239] LustreError: 172110:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5816.774122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.667660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.562521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.497359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.559265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.551706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.451271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.215581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.312250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.138926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.178430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.289601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.245906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.200517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.291271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.361089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.395880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.554256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.354588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.335075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.287160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.364739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.290919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.235481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.994316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.980308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.005255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.960292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.698262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.709944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.834555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.893035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.114868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.296586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.453554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.558200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.654793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.561960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.526193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.531096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.412633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.287770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.087722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.923953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.863069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.825881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.746857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.820674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.687551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.803217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.698787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.739143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.623396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.661672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.681177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.622877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.635540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.677035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.628608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.867603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.961833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.843441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.121633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.258485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.329520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.715151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.949356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.937758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.045940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.050166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.931396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.836491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.749909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.923572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.766878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.762045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.763534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.930411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.126652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.358676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.783880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.047807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.544887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.768743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.093832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.356620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.667308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.022478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.397303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.694340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.032368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.547424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.501973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.998518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.291107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.494775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.627860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.614132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.688225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.749903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.862629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.046664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.246676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.425510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.554679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.488854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.731367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.862924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.133375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.984834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.610221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.331102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.275246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.189929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.419511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.440166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.353267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.435168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.799446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.768688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.665970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.761466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.806840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.944717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.126280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.064245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.963282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.028390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.708527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.669404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.549319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.210580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.034049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.895537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.856831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.735512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.504462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.915187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.172575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.728743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.571866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.444865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.403460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.115489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.384325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.608582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.872624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.423229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.645615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.291798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.530978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.445417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.462421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.257369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.938249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.700239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.579934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.361338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.356307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.443140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.329500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.219956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.001027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.797853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.796639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.841755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.123969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.374901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.295431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.337911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.177600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.128464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.057181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.157694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.199210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.252224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.176683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.142340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.402730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.072307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.742247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.672671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.800246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.767045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.781808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.691594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.956990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.548128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.488638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.244682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.319831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.665704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.534978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.951075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.832919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.225075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.985442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.746960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.399723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.369565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.648851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.010576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.870669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.857664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.903803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.091072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.351928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.015260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.483720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.808725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.200785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.476453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.988496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.775701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.362477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.303808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.078914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.570321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.432773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.597437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.791146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.655567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.590878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.418560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.387551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.350845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.344883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.471407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.685076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.589728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.707574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.565972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.567749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.518621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.476552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.370773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.216969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.250218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.315703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.244492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.416383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.260967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.309276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.368121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.477694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.523350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.670837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.505182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.650546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.600214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.603821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.531046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.722492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.717250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.859889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.909769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.913126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.900398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.911792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.785638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.846893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.796879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.680224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.580708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.396972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.384459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.172481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.138970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.074703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.056353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.877712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.952288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.873279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.848991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.090483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.084796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.426469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.621913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.747290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.784795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.888260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.044597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.037856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.073629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.001790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.221986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.206569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.373400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.562943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.538429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.566433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.579951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.605034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.448162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.521471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.428975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.469069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.463549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.296390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.238456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.422207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.413852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.380817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.290675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.301113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.127489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.923523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.947496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.876549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.913695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.874488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.742035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.962873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.987965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.030697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.970765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.989749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.997157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.049184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.017327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.104480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.867794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.801295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.856426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.787204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.494663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.424683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.349690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.267126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.195745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.111344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.027590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.174887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.974574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.150305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.916201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.929439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.757808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.789532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.669040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.512506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.375501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.153827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.991178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.010061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.815543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.803632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.790296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.767584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.523663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.496049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.247886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.213284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.043654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.842817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.810540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.009711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.898774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.009529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.768777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.538507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.882631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.741839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.542755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.649630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.879357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.106678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.247752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.312530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.520319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.031629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.987693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.786257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.234605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.681370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.300699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.450058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.399790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.251580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.967813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.137042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.041295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.776359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.744524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.730649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.045813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.092773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.219731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.301269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.530805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.790865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.160145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.090758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.633714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.035754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.412441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.559197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.376310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.387844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.483664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.568524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.491849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.545564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.398556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.433621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.511712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.924051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.274703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.531757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.819070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.924886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.485211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.793168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.846228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.142792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.430604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.366723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.900935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.113626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.064154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.304307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.397351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.724093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.946128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.330864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.550315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.935910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.376472] LustreError: 174293:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6635.390258] LustreError: 172110:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7246.820040] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7246.830825] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7246.846400] Lustre: Skipped 457 previous similar messages [ 7246.861354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7247.847952] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7247.855648] Lustre: Skipped 1 previous similar message [ 7252.964715] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7258.084569] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7258.097649] Lustre: Skipped 2 previous similar messages [ 7259.104365] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7259.550839] Lustre: server umount lustre-OST0000 complete [ 7267.812256] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7267.824251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7270.120930] Lustre: server umount lustre-MDT0000 complete [ 7273.453435] LustreError: 173108:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7273.478575] LustreError: 173108:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7274.019901] LustreError: 172072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448920 with bad export cookie 8561806016373205398 [ 7274.023925] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7274.033941] LustreError: 172072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7274.416493] Lustre: server umount lustre-MDT0001 complete [ 7285.650541] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 7288.988735] Key type lgssc unregistered [ 7289.435228] LNet: 227359:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7289.446797] LNetError: 227359:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7289.464947] LNet: Removed LNI 192.168.202.106@tcp [ 7290.574725] Key type .llcrypt unregistered [ 7290.577480] Key type ._llcrypt unregistered [ 7315.121506] Key type ._llcrypt registered [ 7315.128774] Key type .llcrypt registered [ 7315.242182] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7316.928427] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7316.967103] alg: No test for adler32 (adler32-zlib) [ 7318.048540] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7318.301039] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 7319.929445] Key type lgssc registered [ 7321.097807] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7327.673799] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7327.692895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7329.037620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7333.248232] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7338.473174] LustreError: 228775:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7338.488324] LustreError: 228775:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7341.113774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7341.373043] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7345.406577] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7352.655936] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7358.082103] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7364.334169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7364.778416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7371.505380] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7374.017860] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7378.111942] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7383.883801] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7385.807632] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7385.816890] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7385.825911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7385.833161] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7385.835090] LustreError: 228795:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7389.671953] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7389.675104] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7389.704547] Lustre: Skipped 1 previous similar message [ 7391.002736] Lustre: lustre-OST0000: Not available for connect from 192.168.202.6@tcp (stopping) [ 7391.018224] Lustre: Skipped 1 previous similar message [ 7393.847305] Lustre: server umount lustre-OST0000 complete [ 7405.896569] LustreError: 228795:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7427.716627] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7428.022562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7430.056601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7430.072928] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7430.076119] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7433.995649] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7441.193618] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7442.966584] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7448.867529] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b8e03149800.ost_server_uuid 50 [ 7450.550354] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9b8e03149800.ost_server_uuid in FULL state after 0 sec [ 7455.719120] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7455.720102] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7455.727591] LustreError: Skipped 1 previous similar message [ 7455.736600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7455.776843] Lustre: Skipped 1 previous similar message [ 7460.169326] Lustre: server umount lustre-OST0000 complete [ 7464.417726] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7464.422577] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7464.432198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7464.435653] Lustre: Skipped 1 previous similar message [ 7470.548108] Lustre: server umount lustre-MDT0000 complete [ 7471.074422] LustreError: 228772:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7471.100199] LustreError: 228772:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7475.176187] LustreError: 228775:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7475.192098] LustreError: 228775:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7475.239096] LustreError: 233234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449122 with bad export cookie 6999353507571131852 [ 7475.239859] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7475.251092] LustreError: 233234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7475.525864] Lustre: server umount lustre-MDT0001 complete [ 7489.177639] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 7493.513669] Key type lgssc unregistered [ 7493.721931] LNet: 233990:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7493.738915] LNetError: 233990:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7493.749842] LNet: Removed LNI 192.168.202.106@tcp [ 7494.725628] Key type .llcrypt unregistered [ 7494.732294] Key type ._llcrypt unregistered [ 7529.570034] Key type ._llcrypt registered [ 7529.575508] Key type .llcrypt registered [ 7529.700517] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7530.820814] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7530.884969] alg: No test for adler32 (adler32-zlib) [ 7532.261888] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7532.473164] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 7534.121626] Key type lgssc registered [ 7535.476677] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7544.953477] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7544.983507] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7546.736811] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7552.228894] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7557.092338] LustreError: 235719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7557.119237] LustreError: 235719:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7561.366091] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7561.689575] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7566.465578] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7569.600193] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7575.829693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7576.281349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7581.979826] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7591.022530] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7591.202086] Lustre: lustre-OST0001: new disk, initializing [ 7591.206717] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7591.214816] Lustre: 238027:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7591.324632] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7592.414079] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7597.801472] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7598.787129] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7598.816928] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7598.990630] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7610.348313] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7619.069901] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7631.521210] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7631.527420] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7631.542117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7634.916925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7634.923099] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7634.959095] Lustre: Skipped 2 previous similar messages [ 7637.902795] Lustre: server umount lustre-MDT0000 complete [ 7640.041470] LustreError: 237227:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7640.068289] LustreError: 237227:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7641.614399] LustreError: 235702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449288 with bad export cookie 13692834923222554573 [ 7641.617322] LustreError: 236457:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7641.618074] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7641.625397] LustreError: 235702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7641.820717] Lustre: server umount lustre-MDT0001 complete [ 7655.320310] Lustre: server umount lustre-OST0000 complete [ 7669.937383] Lustre: server umount lustre-OST0001 complete [ 7693.787305] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7705.448479] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7706.077228] LustreError: 241144:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7706.194312] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7711.212128] LustreError: 241145:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7711.231920] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7716.332364] LustreError: 241144:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7720.986298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7721.443722] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7726.729468] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7729.945138] Lustre: 242287:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7737.091277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7737.571857] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7744.773723] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7746.820068] LustreError: 242644:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7746.856028] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7752.028765] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7753.354832] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7758.840311] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7759.415419] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7766.954542] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7771.201475] Lustre: 244156:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7788.004439] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7788.018710] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7788.037540] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7788.041765] Lustre: Skipped 3 previous similar messages [ 7791.060322] Lustre: server umount mylustre-MDT0000 complete [ 7794.651850] LustreError: 244357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449441 with bad export cookie 13692834923222557436 [ 7794.667865] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7794.674654] LustreError: 244357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7794.711740] LustreError: 241141:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7794.711966] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7794.730105] LustreError: 241141:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7794.750766] Lustre: Skipped 3 previous similar messages [ 7794.758055] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7794.764512] Lustre: Skipped 4 previous similar messages [ 7795.027947] Lustre: server umount mylustre-MDT0001 complete [ 7801.496928] Lustre: server umount mylustre-OST0000 complete [ 7805.786991] Lustre: server umount mylustre-OST0001 complete [ 7830.596089] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7842.535758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7843.276190] LustreError: 246467:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7843.312128] LustreError: 246467:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7843.432579] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7843.451748] Lustre: Skipped 1 previous similar message [ 7848.981852] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7857.663882] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7861.898898] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7865.340461] Lustre: 247612:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7873.302601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7880.294791] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7882.939290] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7889.414988] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7889.454776] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7895.066257] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7897.084589] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7906.751936] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7910.602116] Lustre: 249481:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7920.611828] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7920.625584] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7920.626947] Lustre: Skipped 4 previous similar messages [ 7920.639201] Lustre: Skipped 3 previous similar messages [ 7926.303961] Lustre: server umount tfs-MDT0000 complete [ 7930.534453] LustreError: 246444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449577 with bad export cookie 13692834923222559613 [ 7930.539914] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7930.547782] LustreError: 246444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7930.854194] LustreError: 246463:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7930.858914] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7930.866861] LustreError: 246463:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 7930.906230] Lustre: Skipped 1 previous similar message [ 7931.222450] Lustre: server umount tfs-MDT0001 complete [ 7938.086342] Lustre: server umount tfs-OST0000 complete [ 7942.334772] Lustre: server umount tfs-OST0001 complete [ 7967.842131] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7979.633049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7980.366040] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7980.377588] Lustre: Skipped 3 previous similar messages [ 7985.211371] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7997.179573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8003.167455] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8006.075980] Lustre: 252874:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8013.084579] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8020.807632] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8022.703027] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8025.800885] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8030.468964] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8036.372962] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8038.073395] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8047.160942] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8051.433472] Lustre: 254744:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8059.750984] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:35:04 (1788449704) [ 8077.284112] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8077.297396] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8077.320626] Lustre: Skipped 3 previous similar messages [ 8077.347972] Lustre: Skipped 7 previous similar messages [ 8082.884266] Lustre: server umount lustre-MDT0000 complete [ 8086.636338] LustreError: 251705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449733 with bad export cookie 13692834923222561531 [ 8086.645623] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8086.648285] LustreError: 251705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8087.523114] LustreError: 251727:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8087.543607] LustreError: 251727:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 8093.422561] Lustre: server umount lustre-MDT0001 complete [ 8103.890310] Lustre: server umount lustre-OST0000 complete [ 8113.840740] Lustre: server umount lustre-OST0001 complete [ 8121.224564] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 8133.876851] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 8170.603559] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8170.603559] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8170.603559] [ 8170.616659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8170.864478] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8170.927672] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8171.005221] Lustre: lustre-MDT0000: new disk, initializing [ 8171.137515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8171.145033] Lustre: Skipped 3 previous similar messages [ 8171.201776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8175.536106] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8185.754090] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8185.754090] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8185.754090] [ 8185.805467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8185.945294] Lustre: 257930:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8186.024388] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8186.029325] Lustre: Skipped 1 previous similar message [ 8186.102588] Lustre: lustre-MDT0001: new disk, initializing [ 8186.280786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8186.298539] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8191.516884] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8202.274860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8202.627905] Lustre: lustre-OST0000: new disk, initializing [ 8202.633069] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8202.645935] Lustre: 259514:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8203.903522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8203.921772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8204.115295] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8209.885862] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8222.405159] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8222.661627] Lustre: lustre-OST0001: new disk, initializing [ 8222.665116] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8222.670765] Lustre: 260386:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8224.403541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8224.411652] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8224.628552] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8230.548436] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8242.774606] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:38:07 (1788449887) [ 8260.959960] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:38:25 (1788449905) [ 8281.058102] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8281.062724] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8281.091743] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8281.091754] Lustre: Skipped 8 previous similar messages [ 8281.127298] Lustre: Skipped 3 previous similar messages [ 8291.296924] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8291.558652] Lustre: server umount lustre-OST0000 complete [ 8302.646398] Lustre: server umount lustre-MDT0000 complete [ 8307.173924] LustreError: 257920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449954 with bad export cookie 13692834923222563309 [ 8307.182439] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8307.186803] LustreError: 257920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8313.732457] Lustre: server umount lustre-MDT0001 complete [ 8325.002804] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 8395.586832] Lustre: server umount lustre-OST0001 complete [ 8402.083775] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 8416.038988] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 8449.981292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8450.242224] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8450.278450] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8450.343183] Lustre: lustre-MDT0000: new disk, initializing [ 8450.459243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8450.462270] Lustre: Skipped 3 previous similar messages [ 8450.493412] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8454.884848] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8465.469850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8465.609295] Lustre: 265181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8466.002046] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8470.683679] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8480.416853] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8486.053302] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8494.083163] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8494.381445] Lustre: lustre-OST0000: new disk, initializing [ 8494.383900] Lustre: Skipped 1 previous similar message [ 8494.399468] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8494.404722] Lustre: Skipped 2 previous similar messages [ 8494.410858] Lustre: 267148:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8495.634517] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8495.651785] Lustre: Skipped 1 previous similar message [ 8495.662332] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8495.807336] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8502.165429] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8512.125843] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8521.189396] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8521.192666] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8521.206958] Lustre: Skipped 5 previous similar messages [ 8521.215631] Lustre: Skipped 13 previous similar messages [ 8524.965666] Lustre: server umount lustre-OST0000 complete [ 8535.935471] Lustre: server umount lustre-MDT0000 complete [ 8536.548274] LustreError: 265191:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8536.576054] LustreError: 265191:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 8540.117167] LustreError: 265173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450187 with bad export cookie 13692834923222564779 [ 8540.122209] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8540.136612] LustreError: 265173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8540.675904] Lustre: server umount lustre-MDT0001 complete [ 8553.993748] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 8557.029053] Key type lgssc unregistered [ 8557.332816] LNet: 268912:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8557.342494] LNetError: 268912:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8557.358992] LNet: Removed LNI 192.168.202.106@tcp [ 8558.407675] Key type .llcrypt unregistered [ 8558.411546] Key type ._llcrypt unregistered [ 8582.792620] Key type ._llcrypt registered [ 8582.799367] Key type .llcrypt registered [ 8582.906854] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 8583.701989] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8583.791366] alg: No test for adler32 (adler32-zlib) [ 8584.804715] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8584.964608] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 8586.592186] Key type lgssc registered [ 8587.587737] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8594.043710] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8594.053110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8595.406753] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8599.602869] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8605.667258] LustreError: 270375:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8605.684606] LustreError: 270375:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8606.233659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8606.480723] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8609.986661] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8616.186584] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8620.866395] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8626.840981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8627.390951] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8633.053282] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8639.407761] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8640.385862] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8640.703985] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8646.114163] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8646.118923] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8646.144135] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8648.324722] Lustre: server umount lustre-OST0000 complete [ 8652.769710] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8652.775406] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8652.785273] Lustre: Skipped 1 previous similar message [ 8652.788432] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8652.790734] Lustre: Skipped 1 previous similar message [ 8656.360436] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8656.361407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8656.370617] Lustre: Skipped 1 previous similar message [ 8658.184747] Lustre: server umount lustre-MDT0000 complete [ 8661.476149] LustreError: 270375:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8661.490254] LustreError: 270375:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8661.841028] LustreError: 270357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450308 with bad export cookie 16606824441381917163 [ 8661.848481] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8661.857837] LustreError: 270357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8662.404531] Lustre: server umount lustre-MDT0001 complete [ 8672.978591] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 8675.571421] Key type lgssc unregistered [ 8675.893774] LNet: 273962:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8675.901401] LNetError: 273962:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8675.924687] LNet: Removed LNI 192.168.202.106@tcp [ 8676.917765] Key type .llcrypt unregistered [ 8676.921605] Key type ._llcrypt unregistered [ 8691.780983] Key type ._llcrypt registered [ 8691.786281] Key type .llcrypt registered [ 8691.899854] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 8693.146805] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8693.172342] alg: No test for adler32 (adler32-zlib) [ 8694.605378] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8695.146567] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 8696.984810] Key type lgssc registered [ 8698.059538] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8707.231668] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8707.252210] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8722.784958] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8727.904486] LustreError: 275136:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.106@tcp: failed processing log, type 4: rc = -110 [ 8754.656669] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8759.646604] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8765.814942] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8767.761880] Lustre: server umount lustre-OST0000 complete [ 8784.702591] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 8787.491289] Key type lgssc unregistered [ 8787.735912] LNet: 276771:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8787.740918] LNetError: 276771:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8787.755920] LNet: Removed LNI 192.168.202.106@tcp [ 8788.562968] Key type .llcrypt unregistered [ 8788.564670] Key type ._llcrypt unregistered [ 8827.252616] Key type ._llcrypt registered [ 8827.255499] Key type .llcrypt registered [ 8827.508808] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 8828.257395] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8828.387184] alg: No test for adler32 (adler32-zlib) [ 8830.186502] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8830.555626] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 8832.296216] Key type lgssc registered [ 8834.259327] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8848.620329] loop0: detected capacity change from 0 to 204800000 [ 8848.865918] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8848.877306] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8849.285501] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8850.666840] loop0: detected capacity change from 0 to 204800000 [ 8850.910083] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8850.938746] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8851.349316] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8852.172496] loop0: detected capacity change from 0 to 204800000 [ 8852.368254] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8852.378794] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8852.446380] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8853.949412] loop0: detected capacity change from 0 to 204800000 [ 8854.218846] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8854.236660] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8854.372901] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8855.414686] loop0: detected capacity change from 0 to 204800000 [ 8855.473820] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8861.206352] loop0: detected capacity change from 0 to 204800000 [ 8861.253044] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8865.976566] loop0: detected capacity change from 0 to 204800000 [ 8865.994127] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8870.175732] loop0: detected capacity change from 0 to 204800000 [ 8870.194150] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8873.808060] loop0: detected capacity change from 0 to 204800000 [ 8873.904533] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8873.956362] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8874.037530] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8874.046154] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8874.054618] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8874.061795] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8874.070034] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8874.078064] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8874.085203] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8874.091060] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8874.099635] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8874.108511] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8874.118622] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8874.125511] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8874.130736] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8874.136668] Lustre: 280187:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8879.176101] Lustre: server umount MGS complete [ 8879.998941] loop0: detected capacity change from 0 to 204800000 [ 8880.082066] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8880.628900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8880.708675] Lustre: lustre-MDT0000: Aborting client recovery [ 8881.648311] loop1: detected capacity change from 0 to 204800000 [ 8881.931376] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8882.033486] Lustre: 280618:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8882.064357] Lustre: 280618:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8882.070367] Lustre: 280618:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8882.084207] Lustre: 280618:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8882.107248] Lustre: 280618:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8882.129492] Lustre: 280618:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8882.153583] Lustre: 280618:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8882.186704] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8882.360447] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8882.421850] Lustre: lustre-MDT0001: Aborting client recovery [ 8883.131295] loop2: detected capacity change from 0 to 204800000 [ 8883.189933] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8883.287801] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8883.293866] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8883.300569] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8883.307512] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8883.317551] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8883.323687] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8883.327600] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8883.431767] Lustre: Found index 0 for lustre-OST0000, updating log [ 8883.459974] Lustre: 280719:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8883.508277] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8883.529444] Lustre: lustre-OST0000: Aborting recovery [ 8884.286639] loop3: detected capacity change from 0 to 204800000 [ 8884.481200] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8884.608186] Lustre: 280785:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8884.630691] Lustre: 280785:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8884.642050] Lustre: 280785:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8884.662991] Lustre: 280785:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8884.674671] Lustre: 280785:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8884.684941] Lustre: 280785:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8884.699153] Lustre: 280785:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8884.714995] Lustre: Found index 1 for lustre-OST0001, updating log [ 8884.772296] Lustre: 280790:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8884.865947] Lustre: lustre-OST0001: Aborting recovery [ 8890.399859] LustreError: 280562:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8891.434541] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8891.446488] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8891.518816] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8891.526332] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8892.461232] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8892.461897] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8893.815927] LustreError: 280529:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2875us: rc = -2 [ 8894.360224] LustreError: 280529:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 676us: rc = -2 [ 8894.370743] LustreError: 280529:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 8907.747162] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8907.751437] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8907.766488] Lustre: Skipped 3 previous similar messages [ 8907.771055] Lustre: Skipped 2 previous similar messages [ 8912.881357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8912.896442] Lustre: Skipped 3 previous similar messages [ 8917.985062] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8917.994801] Lustre: Skipped 3 previous similar messages [ 8921.056185] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8921.286626] Lustre: server umount lustre-MDT0000 complete [ 8922.229319] LustreError: 280513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450569 with bad export cookie 6287381629909638616 [ 8922.234096] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8922.240551] LustreError: 280513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8923.107971] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8923.124161] LustreError: 280526:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8923.124254] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8923.135170] Lustre: Skipped 1 previous similar message [ 8923.166426] LustreError: 280526:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8927.142132] LustreError: 280527:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8927.142678] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8927.180224] Lustre: Skipped 2 previous similar messages [ 8928.629413] Lustre: server umount lustre-MDT0001 complete [ 8935.756587] Lustre: server umount lustre-OST0000 complete [ 8951.264146] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8951.576408] Lustre: server umount lustre-OST0001 complete [ 8959.570865] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:50:04 (1788450604) [ 8991.371927] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 9004.240281] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 9042.988869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9043.266975] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9043.295695] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9043.371484] Lustre: lustre-MDT0000: new disk, initializing [ 9043.492795] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9043.498495] Lustre: Skipped 1 previous similar message [ 9043.515136] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9048.541463] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9060.406136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9060.532400] Lustre: 284859:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9060.576934] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9060.581721] Lustre: Skipped 1 previous similar message [ 9060.659432] Lustre: lustre-MDT0001: new disk, initializing [ 9060.819533] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9060.884524] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9060.906904] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9066.852712] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9077.817495] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9084.348064] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9092.141500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9092.715343] Lustre: lustre-OST0000: new disk, initializing [ 9092.718710] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9092.734042] Lustre: 286830:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9092.858154] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9100.500635] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9101.979328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9101.995947] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9102.150127] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9110.610499] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9116.504974] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9128.561920] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9136.358224] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9144.290187] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9194.473595] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9194.475769] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9194.502068] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9194.514212] LustreError: Skipped 1 previous similar message [ 9208.290363] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9208.498703] Lustre: server umount lustre-OST0000 complete [ 9214.435066] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9214.440208] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9214.447127] Lustre: Skipped 1 previous similar message [ 9214.452682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9214.458303] Lustre: Skipped 5 previous similar messages [ 9218.395353] Lustre: server umount lustre-MDT0000 complete [ 9220.069508] LustreError: 284870:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9220.085433] LustreError: 284870:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9221.824593] LustreError: 284852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450868 with bad export cookie 6287381629909641325 [ 9221.837541] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9221.838441] LustreError: 284852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9222.166669] Lustre: server umount lustre-MDT0001 complete [ 9230.723066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9235.135654] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9239.670996] Lustre: server umount MGS complete [ 9249.525783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9249.936022] LustreError: 290750:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9249.971505] LustreError: 290763:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9250.059900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9255.393737] LustreError: 290751:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9255.405820] LustreError: 290751:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9255.510740] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9263.663422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9269.330876] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9277.274785] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9283.378698] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9290.178134] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9290.496225] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9290.501549] Lustre: Skipped 1 previous similar message [ 9296.030786] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9303.498355] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9329.647329] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9329.670572] Lustre: Skipped 2 previous similar messages [ 9329.675197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9329.677363] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9329.695185] Lustre: Skipped 2 previous similar messages [ 9332.023844] Lustre: server umount lustre-OST0000 complete [ 9339.890663] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9342.810813] Lustre: server umount lustre-MDT0000 complete [ 9344.996206] LustreError: 290751:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9345.003843] LustreError: 290751:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9346.065925] LustreError: 293589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450992 with bad export cookie 6287381629909643278 [ 9346.081758] LustreError: 293589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9346.082937] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9346.333507] Lustre: server umount lustre-MDT0001 complete [ 9361.595446] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 9366.180806] Key type lgssc unregistered [ 9366.904845] LNet: 294699:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9366.926943] LNetError: 294699:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9367.005852] LNet: Removed LNI 192.168.202.106@tcp [ 9368.587546] Key type .llcrypt unregistered [ 9368.590470] Key type ._llcrypt unregistered [ 9413.513675] Key type ._llcrypt registered [ 9413.516615] Key type .llcrypt registered [ 9413.676531] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 9432.609464] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 9434.397912] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9434.471105] alg: No test for adler32 (adler32-zlib) [ 9435.609687] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9435.793261] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 9437.464917] Key type lgssc registered [ 9438.938453] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9475.353190] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9475.391276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9476.643440] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9476.689530] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9476.804437] Lustre: lustre-MDT0000: new disk, initializing [ 9476.950900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9476.969717] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9482.166469] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9494.062594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9494.227400] Lustre: 298658:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9494.268728] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9494.274184] Lustre: Skipped 1 previous similar message [ 9494.360216] Lustre: lustre-MDT0001: new disk, initializing [ 9494.461607] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9494.506148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9494.520221] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9500.099324] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9509.784251] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9515.749723] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9522.859984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9523.133214] Lustre: lustre-OST0000: new disk, initializing [ 9523.139219] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9523.150735] Lustre: 300626:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9523.301372] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9528.785730] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9530.434973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9530.441059] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9530.558703] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9537.943135] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9546.303615] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9556.462857] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9559.053308] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9563.521510] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9602.531228] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9602.542358] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9602.545468] Lustre: Skipped 1 previous similar message [ 9602.546592] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9606.501109] Lustre: server umount lustre-OST0000 complete [ 9612.259162] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9612.269508] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9612.292071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9612.299122] Lustre: Skipped 1 previous similar message [ 9617.352147] Lustre: server umount lustre-MDT0000 complete [ 9621.194663] LustreError: 298649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451268 with bad export cookie 13696634641937234455 [ 9621.196395] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9621.202803] LustreError: 298649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9621.649174] Lustre: server umount lustre-MDT0001 complete [ 9631.715677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9637.309679] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9643.097692] Lustre: server umount MGS complete [ 9656.519370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9657.246222] LustreError: 304361:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9657.271397] LustreError: 304361:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9657.283964] LustreError: 304374:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9657.354198] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9662.443310] LustreError: 304362:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9663.110981] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9667.554910] LustreError: 304361:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9672.677989] LustreError: 304362:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9673.125939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9673.661448] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9678.964817] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9687.392676] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9693.562339] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9701.213968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9701.651302] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9708.739926] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9715.773070] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9744.869991] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9744.872294] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9744.899266] Lustre: Skipped 1 previous similar message [ 9744.899955] Lustre: Skipped 2 previous similar messages [ 9749.990845] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9749.998914] Lustre: Skipped 2 previous similar messages [ 9750.794037] Lustre: server umount lustre-OST0000 complete [ 9755.109813] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9755.120769] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9755.126685] Lustre: Skipped 1 previous similar message [ 9755.140905] Lustre: Skipped 1 previous similar message [ 9761.233887] Lustre: server umount lustre-MDT0000 complete [ 9764.978211] LustreError: 304342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451411 with bad export cookie 13696634641937236282 [ 9764.984876] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9764.992817] LustreError: 304342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9765.623581] Lustre: server umount lustre-MDT0001 complete [ 9779.501414] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 9782.389738] Key type lgssc unregistered [ 9782.731453] LNet: 308312:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9782.740224] LNetError: 308312:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9782.750466] LNet: Removed LNI 192.168.202.106@tcp [ 9783.692822] Key type .llcrypt unregistered [ 9783.696703] Key type ._llcrypt unregistered [ 9806.591505] Key type ._llcrypt registered [ 9806.594262] Key type .llcrypt registered [ 9806.711706] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 9823.290275] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 9824.134358] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9824.238955] alg: No test for adler32 (adler32-zlib) [ 9825.896884] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9826.281375] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 9827.936175] Key type lgssc registered [ 9829.881336] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9859.583816] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 9873.298968] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9873.320595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9874.628531] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9874.713211] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9874.811200] Lustre: lustre-MDT0000: new disk, initializing [ 9874.897376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9874.916623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9879.844967] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 9893.092908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9893.501442] Lustre: lustre-OST0000: new disk, initializing [ 9893.510552] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9893.515210] Lustre: Skipped 1 previous similar message [ 9893.520249] Lustre: 312365:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9893.612986] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9900.763724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9900.777662] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9900.802415] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9901.331176] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10715.106224] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10715.135637] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10717.670568] Lustre: server umount lustre-MDT0000 complete [10721.070955] LustreError: 311552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452368 with bad export cookie 13483356692435503437 [10721.077017] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10721.077307] LustreError: 311552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10721.317399] Lustre: server umount lustre-OST0000 complete [10745.082784] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [10756.461525] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [10793.814521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10794.030950] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10794.053618] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10794.122396] Lustre: lustre-MDT0000: new disk, initializing [10794.203608] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10794.218726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10798.580197] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10809.215550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10809.333714] Lustre: 316696:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10809.388517] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10809.391951] Lustre: Skipped 1 previous similar message [10809.493965] Lustre: lustre-MDT0001: new disk, initializing [10809.698752] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10809.741295] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10809.752352] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10815.968542] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10826.542602] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10833.797870] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10840.646536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10841.025863] Lustre: lustre-OST0000: new disk, initializing [10841.029222] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10841.037709] Lustre: 318664:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10841.155195] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10847.444623] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10848.822408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10848.836207] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10848.913130] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10857.134660] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10862.049794] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10862.368308] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10867.122451] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10867.414637] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10869.729087] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10869.733932] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10869.742465] LustreError: Skipped 1 previous similar message [10869.745275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10869.779061] Lustre: Skipped 1 previous similar message [10874.870043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10874.879530] Lustre: Skipped 2 previous similar messages [10875.883687] Lustre: server umount lustre-OST0000 complete [10879.979254] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10879.998748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10880.012076] Lustre: Skipped 1 previous similar message [10885.089484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10885.100108] Lustre: Skipped 1 previous similar message [10894.304138] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10894.470454] Lustre: server umount lustre-MDT0000 complete [10898.151453] LustreError: 317489:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452545 with bad export cookie 13483356692438444256 [10898.155548] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10898.161858] LustreError: 317489:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10898.508858] Lustre: server umount lustre-MDT0001 complete [10901.958744] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10903.768232] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 12:22:28 (1788452548) [10911.546736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10912.117599] LustreError: 320977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10912.143107] LustreError: 320977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10912.196042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10916.124880] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10917.358323] LustreError: 320978:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10922.465222] LustreError: 320977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10925.764435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10926.265428] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10931.058936] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10938.687063] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10944.865973] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10951.288688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10951.583955] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10957.198643] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10965.439688] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10971.290225] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10971.450247] Lustre: lustre-OST0001: new disk, initializing [10971.454667] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10971.460977] Lustre: 323813:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10971.548270] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10978.614792] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [10979.546167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10979.570043] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10979.628819] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10979.645256] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10980.713135] LustreError: 324197:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10980.724517] LustreError: 324197:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10981.739483] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10983.779416] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10983.788818] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10987.941894] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10987.947515] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10989.045349] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10989.394955] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10995.409155] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b8e06412800.ost_server_uuid 50 [10996.271395] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10996.287945] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10997.544670] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9b8e06412800.ost_server_uuid in FULL state after 0 sec [11003.463408] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9b8e06412800.ost_server_uuid 50 [11005.691664] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9b8e06412800.ost_server_uuid in FULL state after 0 sec [11007.783166] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11023.329333] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11023.332329] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11023.348440] LustreError: Skipped 1 previous similar message [11023.351066] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11023.370706] Lustre: Skipped 2 previous similar messages [11023.391511] Lustre: Skipped 3 previous similar messages [11028.684050] Lustre: server umount lustre-OST0001 complete [11029.756933] LustreError: 322798:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11036.142077] LustreError: 322799:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11036.145940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11036.182628] LustreError: 322799:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11041.253825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11041.265072] Lustre: Skipped 6 previous similar messages [11041.701463] Lustre: server umount lustre-OST0000 complete [11046.369660] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11046.385103] Lustre: Skipped 2 previous similar messages [11051.491424] LustreError: 322000:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11051.507916] LustreError: 322000:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11051.627325] Lustre: server umount lustre-MDT0000 complete [11056.271388] LustreError: 320956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452703 with bad export cookie 13483356692438445131 [11056.280195] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11056.726648] Lustre: server umount lustre-MDT0001 complete [11069.290738] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [11072.275438] Key type lgssc unregistered [11072.620717] LNet: 326508:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11072.642572] LNetError: 326508:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11072.663721] LNet: Removed LNI 192.168.202.106@tcp [11073.811699] Key type .llcrypt unregistered [11073.814204] Key type ._llcrypt unregistered [11101.898957] Key type ._llcrypt registered [11101.901152] Key type .llcrypt registered [11102.019121] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [11103.692577] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11103.768435] alg: No test for adler32 (adler32-zlib) [11105.009920] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11105.324519] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [11107.008807] Key type lgssc registered [11108.087868] Lustre: Echo OBD driver; http://www.lustre.org/ [11115.966387] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11116.006751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11117.597640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11122.546966] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11127.791085] LustreError: 327930:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11127.814515] LustreError: 327930:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11130.189629] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11132.912890] LustreError: 327931:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11137.805371] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11138.143414] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11143.038984] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11153.321677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11153.646075] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11159.926213] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11162.858628] LustreError: 329611:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11162.891592] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11166.496648] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11168.235693] LustreError: 330099:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11168.250622] LustreError: 330099:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11173.359883] LustreError: 329611:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11173.402796] LustreError: 329611:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11174.762593] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11175.135536] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11180.554570] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11181.822290] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11189.394131] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11195.597683] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11195.856684] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11298.278736] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11298.279934] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11298.309041] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11303.415953] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11303.428574] Lustre: Skipped 2 previous similar messages [11303.722311] Lustre: server umount lustre-OST0001 complete [11308.514681] LustreError: 330906:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11308.552849] LustreError: 330906:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11311.588069] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11311.588923] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11311.597984] LustreError: Skipped 1 previous similar message [11311.598750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11311.633924] Lustre: Skipped 2 previous similar messages [11313.640452] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11313.645489] Lustre: Skipped 2 previous similar messages [11316.542637] Lustre: server umount lustre-OST0000 complete [11322.337588] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11322.341554] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11322.381482] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11326.838880] Lustre: server umount lustre-MDT0000 complete [11329.000038] LustreError: 327931:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11329.015515] LustreError: 327931:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11331.080456] LustreError: 331616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452978 with bad export cookie 18139423082573496055 [11331.082197] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11331.090975] LustreError: 331616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11331.339577] Lustre: server umount lustre-MDT0001 complete [11345.481979] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [11348.669097] Key type lgssc unregistered [11348.946854] LNet: 332876:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11348.953990] LNetError: 332876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11348.966027] LNet: Removed LNI 192.168.202.106@tcp [11350.111640] Key type .llcrypt unregistered [11350.114858] Key type ._llcrypt unregistered [11393.846965] Key type ._llcrypt registered [11393.849232] Key type .llcrypt registered [11394.073699] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [11395.602149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11395.615172] alg: No test for adler32 (adler32-zlib) [11396.665898] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11396.984625] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [11398.696314] Key type lgssc registered [11400.455772] Lustre: Echo OBD driver; http://www.lustre.org/ [11412.973376] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11413.024340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11415.878262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11421.369930] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11425.268601] LustreError: 335169:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11425.295846] LustreError: 335169:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11430.388826] LustreError: 335168:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11432.830967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11433.261780] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11438.378345] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11441.130717] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11447.904675] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11448.310031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11454.660286] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11458.537810] LustreError: 336657:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11460.591848] LustreError: 336657:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11460.612368] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11460.619625] LustreError: 336657:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11464.059437] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11464.368735] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11469.858963] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11471.145255] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11480.885334] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11490.076830] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11505.121436] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11505.129216] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11505.146331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11505.639950] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11505.654141] Lustre: Skipped 1 previous similar message [11505.659536] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11505.664041] Lustre: Skipped 2 previous similar messages [11507.656264] Lustre: server umount lustre-MDT0000 complete [11510.760392] LustreError: 335164:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11510.788607] LustreError: 335164:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11511.347897] LustreError: 335148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453158 with bad export cookie 18278113510967115452 [11511.357504] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11511.364403] LustreError: 335148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11511.792675] Lustre: server umount lustre-MDT0001 complete [11525.192784] Lustre: server umount lustre-OST0000 complete [11538.356763] Lustre: server umount lustre-OST0001 complete [11554.617436] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [11564.399603] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11564.826249] LustreError: 340436:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11564.914770] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11569.671863] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11577.586589] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11577.943419] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11583.021829] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11586.093200] Lustre: 341578:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11592.293280] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11598.023930] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11604.014461] LustreError: 341933:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11604.033155] LustreError: 341933:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11605.888530] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11606.138267] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11606.150551] Lustre: Skipped 1 previous similar message [11609.207417] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11609.221324] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11613.193435] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11622.206616] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11626.311774] Lustre: 343449:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11634.146051] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11634.159781] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11634.182586] Lustre: Skipped 1 previous similar message [11634.192988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11637.687730] Lustre: server umount lustre-MDT0000 complete [11640.301343] LustreError: 340436:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11640.324985] LustreError: 340436:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11642.229285] LustreError: 343451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453289 with bad export cookie 18278113510967117153 [11642.234182] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11642.246637] LustreError: 343451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11643.092347] Lustre: server umount lustre-MDT0001 complete [11657.971801] Lustre: server umount lustre-OST0000 complete [11661.537703] Lustre: 334428:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788453292/real 1788453292] req@ffff99bf8d892680 x1875328938131456/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788453308 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11661.588149] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11661.609613] Lustre: Skipped 3 previous similar messages [11662.645063] Lustre: server umount lustre-OST0001 complete [11665.670533] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11667.094265] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11668.631242] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:35:13 (1788453313) [11689.985859] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [11697.916707] loop0: detected capacity change from 0 to 1099511627776 [11698.541150] SGI XFS with ACLs, security attributes, quota, no debug enabled [11698.546812] *** VALIDATE xfs *** [11698.577547] XFS (loop0): Mounting V5 Filesystem [11698.609614] XFS (loop0): Ending clean mount [11699.645895] loop1: detected capacity change from 0 to 18691697672192 [11810.986209] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [11821.048145] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [11855.872889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11856.096620] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11856.124059] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11856.209294] Lustre: lustre-MDT0000: new disk, initializing [11856.265035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11856.277615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11860.970455] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11871.154380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11871.256671] Lustre: 348585:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11871.313460] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11871.319275] Lustre: Skipped 1 previous similar message [11871.378289] Lustre: lustre-MDT0001: new disk, initializing [11871.490309] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11871.504429] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11875.740219] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11884.046985] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11889.841733] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11896.635949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11896.967275] Lustre: lustre-OST0000: new disk, initializing [11896.971242] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11896.976199] Lustre: 350553:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11898.883023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11898.893572] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11899.024757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11903.337396] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11912.049034] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11916.564978] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11916.724610] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11921.813585] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11922.005840] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11924.456870] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11924.460659] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11924.468791] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11924.484992] Lustre: Skipped 1 previous similar message [11924.506213] Lustre: Skipped 5 previous similar messages [11929.576967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11929.595068] Lustre: Skipped 1 previous similar message [11938.275628] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11938.639812] Lustre: server umount lustre-OST0000 complete [11943.394352] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11943.398961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11943.416121] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11943.420246] Lustre: Skipped 2 previous similar messages [11950.050918] LustreError: 349467:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11950.090582] LustreError: 349467:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11950.205530] Lustre: server umount lustre-MDT0000 complete [11955.827703] LustreError: 349380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453602 with bad export cookie 18278113510967118707 [11955.828531] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11955.832524] LustreError: 349380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11956.917635] Lustre: server umount lustre-MDT0001 complete [11966.046217] XFS (loop0): Unmounting Filesystem [11970.974547] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:40:15 (1788453615) [11978.996730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11979.561036] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11979.571077] Lustre: Skipped 2 previous similar messages [11984.326364] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [11992.558179] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11997.473799] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12005.754092] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12012.304639] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12020.533462] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12028.583466] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12035.722843] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12047.852720] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12047.862725] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12047.874477] Lustre: Skipped 1 previous similar message [12047.876201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12047.876208] Lustre: Skipped 2 previous similar messages [12053.773319] Lustre: server umount lustre-OST0000 complete [12063.946496] Lustre: server umount lustre-MDT0000 complete [12068.088981] LustreError: 353092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453715 with bad export cookie 18278113510967119589 [12068.090829] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12068.099807] LustreError: 353092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12068.383696] Lustre: server umount lustre-MDT0001 complete [12079.833861] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [12083.043224] Key type lgssc unregistered [12083.467973] LNet: 356745:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12083.481569] LNetError: 356745:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12084.523764] LNet: Removed LNI 192.168.202.106@tcp [12085.671168] Key type .llcrypt unregistered [12085.674797] Key type ._llcrypt unregistered [12113.798730] Key type ._llcrypt registered [12113.801246] Key type .llcrypt registered [12113.936785] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [12115.929354] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12115.943535] alg: No test for adler32 (adler32-zlib) [12117.142481] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [12117.453353] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [12119.184375] Key type lgssc registered [12120.724249] Lustre: Echo OBD driver; http://www.lustre.org/ [12131.511581] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12131.529140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12132.857342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12137.707981] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12143.080238] LustreError: 358316:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12143.097821] LustreError: 358316:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12145.970337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12146.527761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12151.984435] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12160.748470] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12167.404869] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12174.239308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12174.646335] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12181.024751] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12187.973661] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12201.227200] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.6@tcp (stopping) [12202.468549] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12202.484467] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12205.238662] Lustre: server umount lustre-MDT0001 complete [12206.328466] LustreError: 358311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12207.594705] LustreError: 358317:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12207.620890] LustreError: 358317:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12211.453376] LustreError: 358312:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12215.552511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12215.688920] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12215.694531] Lustre: Found index 1 for lustre-MDT0001, updating log [12215.757253] Lustre: 358306:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12216.037765] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12220.560606] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12220.934957] LustreError: 361783:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12220.940402] LustreError: 361783:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12220.948229] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12220.986058] LustreError: 361789:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.106@tcp: cfg command failed: rc = -17 [12220.994539] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12220.994539] [12221.431921] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12231.649026] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12231.658198] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12231.679203] Lustre: Skipped 1 previous similar message [12231.682105] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12231.702825] Lustre: Skipped 2 previous similar messages [12236.784880] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12236.788063] Lustre: Skipped 1 previous similar message [12237.044203] LustreError: 359961:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12237.065958] LustreError: 359961:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12237.142516] Lustre: server umount lustre-MDT0001 complete [12247.015119] LustreError: 358313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12247.032566] LustreError: 358313:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12248.153055] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12248.243655] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12248.247899] Lustre: Found index 1 for lustre-MDT0001, updating log [12248.290946] Lustre: 358306:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12248.677849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12253.211418] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12253.689330] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12256.747965] LustreError: 362991:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12256.756498] LustreError: 362991:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12256.764972] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12256.769053] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12256.828036] LustreError: 362994:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12256.834484] LustreError: 362994:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12256.838890] LustreError: 362994:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.106@tcp: cfg command failed: rc = -17 [12256.849323] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.106@tcp [12256.849323] [12558.611245] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.6@tcp (stopping) [12560.867659] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12560.873306] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12560.894895] Lustre: Skipped 1 previous similar message [12564.732708] Lustre: server umount lustre-MDT0001 complete [12565.994045] LustreError: 358316:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12566.023263] LustreError: 358316:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12573.843394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12573.919769] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12573.924522] Lustre: Found index 1 for lustre-MDT0001, updating log [12573.952232] Lustre: 358306:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12574.200395] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12578.581253] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [12579.349223] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12581.379783] LustreError: 371277:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12581.398381] LustreError: 371277:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12581.430781] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12581.442592] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12581.527564] LustreError: 371285:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12581.569084] LustreError: 371285:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.106@tcp: cfg command failed: rc = -114 [12581.588395] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12581.588395] [12901.858945] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12901.861622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12901.891705] Lustre: Skipped 1 previous similar message [12901.917423] Lustre: Skipped 5 previous similar messages [12908.018575] Lustre: server umount lustre-MDT0000 complete [12911.703026] LustreError: 360159:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788454558 with bad export cookie 9371914845677013807 [12911.714353] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12911.718324] LustreError: 360159:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12912.101770] LustreError: 358311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12912.111588] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12912.118135] LustreError: 358311:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12912.164632] Lustre: Skipped 1 previous similar message [12918.180325] Lustre: server umount lustre-MDT0001 complete [12922.613228] Lustre: server umount lustre-OST0000 complete [12933.515987] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [12943.769426] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [12980.098399] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:57:04 (1788454624) [12987.840546] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12988.428276] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12988.504453] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12988.561434] Lustre: lustre-MDT0000: new disk, initializing [12988.635604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12988.642978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12994.362512] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [13003.832465] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13004.027707] Lustre: 381696:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13004.128632] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13004.132909] Lustre: Skipped 1 previous similar message [13004.291339] Lustre: lustre-MDT0001: new disk, initializing [13004.510917] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13004.627914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13004.635459] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13009.597530] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [13019.808689] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13024.889443] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13033.110558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13033.464765] Lustre: lustre-OST0000: new disk, initializing [13033.467996] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13033.476921] Lustre: 383665:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13033.629527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13042.901946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13042.943316] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13043.213042] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13043.670281] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [13054.765983] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13063.650229] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13063.660181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13063.663020] Lustre: Skipped 1 previous similar message [13063.684064] Lustre: Skipped 7 previous similar messages [13067.528295] Lustre: server umount lustre-MDT0000 complete [13068.772059] LustreError: 383673:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13068.802678] LustreError: 383673:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13070.795144] LustreError: 381690:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788454717 with bad export cookie 9371914845677015760 [13070.803908] LustreError: 382503:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13070.808685] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13071.130249] Lustre: server umount lustre-MDT0001 complete [13085.543502] Lustre: server umount lustre-OST0000 complete [13119.263482] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [13132.288411] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [13170.129118] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12836 sec ========= 13:00:14 (1788454814) [13172.441509] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 13:00:16 (1788454816) === [13176.530706] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 13:00:21 (1788454821) === [13221.174330] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [13225.281372] Key type lgssc unregistered [13225.640421] LNet: 390049:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13225.644672] LNetError: 390049:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13225.682487] LNet: Removed LNI 192.168.202.106@tcp [13226.650111] Key type .llcrypt unregistered [13226.653064] Key type ._llcrypt unregistered