[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 494452236 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002390] x2apic enabled [ 0.003007] Switched APIC routing to physical x2apic. [ 0.004012] kvm-guest: setup PV IPIs [ 0.006750] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.007026] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.008018] pid_max: default: 32768 minimum: 301 [ 0.009141] LSM: Security Framework initializing [ 0.010061] Yama: becoming mindful. [ 0.011043] SELinux: Initializing. [ 0.012091] *** VALIDATE selinux *** [ 0.020924] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025117] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026134] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027110] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028137] *** VALIDATE tmpfs *** [ 0.030426] *** VALIDATE proc *** [ 0.031261] *** VALIDATE cgroup *** [ 0.032011] *** VALIDATE cgroup2 *** [ 0.034059] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035166] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036013] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037030] Spectre V2 : User space: Vulnerable [ 0.038011] Speculative Store Bypass: Vulnerable [ 0.041535] debug: unmapping init [mem 0xffffffffb3459000-0xffffffffb3460fff] [ 0.043176] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044698] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045028] ... version: 2 [ 0.046014] ... bit width: 48 [ 0.047015] ... generic registers: 4 [ 0.048015] ... value mask: 0000ffffffffffff [ 0.049017] ... max period: 00007fffffffffff [ 0.050015] ... fixed-purpose events: 3 [ 0.051013] ... event mask: 000000070000000f [ 0.052365] rcu: Hierarchical SRCU implementation. [ 0.054435] smp: Bringing up secondary CPUs ... [ 0.055635] x86: Booting SMP configuration: [ 0.056035] .... node #0, CPUs: #1 #2 #3 [ 0.060027] smp: Brought up 1 node, 4 CPUs [ 0.062018] smpboot: Max logical packages: 1 [ 0.063016] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.296038] node 0 deferred pages initialised in 227ms [ 0.302129] devtmpfs: initialized [ 0.303268] x86/mm: Memory block size: 128MB [ 0.305565] gcov: version magic: 0x41383552 [ 0.307382] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.308113] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.309242] pinctrl core: initialized pinctrl subsystem [ 0.310175] [ 0.310793] ************************************************************* [ 0.311017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.312017] ** ** [ 0.313017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.314020] ** ** [ 0.315019] ** This means that this kernel is built to expose internal ** [ 0.316017] ** IOMMU data structures, which may compromise security on ** [ 0.317019] ** your system. ** [ 0.318014] ** ** [ 0.319017] ** If you see this message and you are not debugging the ** [ 0.320022] ** kernel, report this immediately to your vendor! ** [ 0.321019] ** ** [ 0.322017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.323019] ************************************************************* [ 0.324604] NET: Registered protocol family 16 [ 0.325487] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.326064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.327071] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.328671] cpuidle: using governor menu [ 0.330664] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.333518] PCI: Using configuration type 1 for base access [ 0.336132] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.345056] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.347048] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.350141] cryptd: max_cpu_qlen set to 1000 [ 0.353294] ACPI: Added _OSI(Module Device) [ 0.354023] ACPI: Added _OSI(Processor Device) [ 0.356019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.357016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.362725] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.369651] ACPI: Interpreter enabled [ 0.371138] ACPI: PM: (supports S0 S3 S4 S5) [ 0.372014] ACPI: Using IOAPIC for interrupt routing [ 0.374189] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.376515] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.387000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.390061] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.393021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.395110] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.400612] acpiphp: Slot [2] registered [ 0.401172] acpiphp: Slot [5] registered [ 0.403158] acpiphp: Slot [6] registered [ 0.404147] acpiphp: Slot [7] registered [ 0.405126] acpiphp: Slot [8] registered [ 0.407115] acpiphp: Slot [9] registered [ 0.408194] acpiphp: Slot [10] registered [ 0.409142] acpiphp: Slot [3] registered [ 0.410097] acpiphp: Slot [4] registered [ 0.411087] acpiphp: Slot [11] registered [ 0.412093] acpiphp: Slot [12] registered [ 0.413116] acpiphp: Slot [13] registered [ 0.415141] acpiphp: Slot [14] registered [ 0.416139] acpiphp: Slot [15] registered [ 0.418134] acpiphp: Slot [16] registered [ 0.419108] acpiphp: Slot [17] registered [ 0.420117] acpiphp: Slot [18] registered [ 0.421120] acpiphp: Slot [19] registered [ 0.423136] acpiphp: Slot [20] registered [ 0.424127] acpiphp: Slot [21] registered [ 0.425093] acpiphp: Slot [22] registered [ 0.427139] acpiphp: Slot [23] registered [ 0.428121] acpiphp: Slot [24] registered [ 0.429095] acpiphp: Slot [25] registered [ 0.430097] acpiphp: Slot [26] registered [ 0.431129] acpiphp: Slot [27] registered [ 0.432123] acpiphp: Slot [28] registered [ 0.434120] acpiphp: Slot [29] registered [ 0.435111] acpiphp: Slot [30] registered [ 0.436105] acpiphp: Slot [31] registered [ 0.437122] PCI host bridge to bus 0000:00 [ 0.438024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.440033] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.442028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.444030] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.446060] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.450066] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.452287] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.456157] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.459491] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.470011] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.476047] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.479032] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.481032] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.483030] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.487713] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.491035] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.494056] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.497822] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.505024] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.518814] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.523989] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.531456] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.538026] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.544024] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.562033] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.572473] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.584018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.594025] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.614019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.627720] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.634027] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.646034] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.680030] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.698183] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.706016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.715016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.738015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.751380] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.766027] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.773026] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.793022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.807387] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.815023] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.822017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.846026] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.864734] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.867520] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.870417] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.873421] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.875261] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.882046] iommu: Default domain type: Passthrough [ 0.884538] SCSI subsystem initialized [ 0.885162] ACPI: bus type USB registered [ 0.887176] usbcore: registered new interface driver usbfs [ 0.889116] usbcore: registered new interface driver hub [ 0.891127] usbcore: registered new device driver usb [ 0.893278] pps_core: LinuxPPS API ver. 1 registered [ 0.895019] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.898102] PTP clock support registered [ 0.900177] EDAC MC: Ver: 3.0.0 [ 0.902164] PCI: Using ACPI for IRQ routing [ 0.905213] NetLabel: Initializing [ 0.907022] NetLabel: domain hash size = 128 [ 0.908012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.911116] NetLabel: unlabeled traffic allowed by default [ 0.913144] vgaarb: loaded [ 0.915313] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.916011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.928417] clocksource: Switched to clocksource kvm-clock [ 1.040698] VFS: Disk quotas dquot_6.6.0 [ 1.042679] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.045712] *** VALIDATE ramfs *** [ 1.047306] *** VALIDATE hugetlbfs *** [ 1.048987] pnp: PnP ACPI init [ 1.051624] pnp: PnP ACPI: found 6 devices [ 1.081289] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.084912] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.087591] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.090100] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.092754] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.095150] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.098261] NET: Registered protocol family 2 [ 1.101051] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.107993] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.113613] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.119125] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.122921] TCP: Hash tables configured (established 65536 bind 65536) [ 1.126150] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.129499] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.132583] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.135671] NET: Registered protocol family 1 [ 1.140536] RPC: Registered named UNIX socket transport module. [ 1.143278] RPC: Registered udp transport module. [ 1.145132] RPC: Registered tcp transport module. [ 1.146978] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.149473] NET: Registered protocol family 44 [ 1.151421] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.153979] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.156574] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.159296] PCI: CLS 0 bytes, default 64 [ 1.160894] Unpacking initramfs... [ 2.564464] debug: unmapping init [mem 0xffff96e7fcc54000-0xffff96e7fffbffff] [ 2.568868] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.571319] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.574260] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 3.073292] Initialise system trusted keyrings [ 3.075296] Key type blacklist registered [ 3.077388] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.110362] zbud: loaded [ 3.117049] *** VALIDATE nfs *** [ 3.118493] *** VALIDATE nfs4 *** [ 3.121109] pstore: using deflate compression [ 3.125366] Platform Keyring initialized [ 3.254644] NET: Registered protocol family 38 [ 3.256732] Key type asymmetric registered [ 3.258343] Asymmetric key parser 'x509' registered [ 3.260573] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.263400] io scheduler mq-deadline registered [ 3.265594] io scheduler kyber registered [ 3.267632] io scheduler bfq registered [ 3.269729] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.273326] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.276819] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.279961] ACPI: Power Button [PWRF] [ 3.287738] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.295714] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.315346] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.325923] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.348434] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.375070] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.403352] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.408556] Non-volatile memory driver v1.3 [ 3.410409] Linux agpgart interface v0.103 [ 3.450834] virtio_blk virtio1: [vda] 145168 512-byte logical blocks (74.3 MB/70.9 MiB) [ 3.454416] vda: detected capacity change from 0 to 74326016 [ 3.474130] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.477382] vdb: detected capacity change from 0 to 1073741824 [ 3.492966] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.496199] vdc: detected capacity change from 0 to 2621440000 [ 3.512421] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.515985] vdd: detected capacity change from 0 to 2621440000 [ 3.534053] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.536986] vde: detected capacity change from 0 to 4294967296 [ 3.554251] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.557044] vdf: detected capacity change from 0 to 4294967296 [ 3.565093] libphy: Fixed MDIO Bus: probed [ 3.571598] usbcore: registered new interface driver usbserial_generic [ 3.574397] usbserial: USB Serial support registered for generic [ 3.577122] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.581391] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.583458] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.586523] mousedev: PS/2 mouse device common for all mice [ 3.589664] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.593542] rtc_cmos 00:05: RTC can wake from S4 [ 3.596518] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.600226] rtc_cmos 00:05: registered as rtc0 [ 3.602831] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.602964] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.605909] intel_pstate: CPU model not supported [ 3.612454] hid: raw HID events driver (C) Jiri Kosina [ 3.614875] usbcore: registered new interface driver usbhid [ 3.616921] usbhid: USB HID core driver [ 3.618496] drop_monitor: Initializing network drop monitor service [ 3.621056] Initializing XFRM netlink socket [ 3.623054] NET: Registered protocol family 10 [ 3.626478] Segment Routing with IPv6 [ 3.628108] NET: Registered protocol family 17 [ 3.630097] mpls_gso: MPLS GSO support [ 3.635941] RAS: Correctable Errors collector initialized. [ 3.638214] AVX version of gcm_enc/dec engaged. [ 3.640036] AES CTR mode by8 optimization enabled [ 3.731237] sched_clock: Marking stable (3731213248, 0)->(4744273230, -1013059982) [ 3.735218] registered taskstats version 1 [ 3.737777] Loading compiled-in X.509 certificates [ 3.739879] zswap: loaded using pool lzo/zbud [ 3.767078] Key type big_key registered [ 3.780579] Key type encrypted registered [ 3.782423] ima: No TPM chip found, activating TPM-bypass! [ 3.784653] ima: Allocated hash algorithm: sha1 [ 3.786607] ima: No architecture policies found [ 3.788459] evm: Initialising EVM extended attributes: [ 3.790471] evm: security.selinux [ 3.791871] evm: security.ima [ 3.793056] evm: security.capability [ 3.794399] evm: HMAC attrs: 0x1 [ 3.797097] rtc_cmos 00:05: setting system clock to 2026-06-23 13:11:11 UTC (1782220271) [ 3.803480] debug: unmapping init [mem 0xffffffffb4403000-0xffffffffb45fffff] [ 3.806522] debug: unmapping init [mem 0xffffffffb3182000-0xffffffffb3458fff] [ 3.817208] Write protecting the kernel read-only data: 28672k [ 3.820855] debug: unmapping init [mem 0xffffffffb1803000-0xffffffffb19fffff] [ 3.823624] debug: unmapping init [mem 0xffffffffb2114000-0xffffffffb21fffff] [ 3.861395] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.869912] systemd[1]: Detected virtualization kvm. [ 3.872073] systemd[1]: Detected architecture x86-64. [ 3.874176] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.901181] systemd[1]: No hostname configured. [ 3.903076] systemd[1]: Set hostname to . [ 3.905164] random: systemd: uninitialized urandom read (16 bytes read) [ 3.907809] systemd[1]: Initializing machine ID from random generator. [ 3.949429] random: ln: uninitialized urandom read (6 bytes read) [ 4.053791] random: systemd: uninitialized urandom read (16 bytes read) [ 4.056676] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 4.062354] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 4.066971] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.791559] device-mapper: uevent: version 1.0.3 [ 4.794450] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.602771] virtio_net virtio0 ens2: renamed from eth0 [ 5.607063] random: fast init done [ 5.772172] scsi host0: ata_piix [ 5.786291] scsi host1: ata_piix [ 5.789153] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.791457] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.205042] dracut-initqueue[591]: RTNETLINK answers: File exists [ 10.299442] random: crng init done [ 10.301120] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.872137] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.183992] printk: systemd: 22 output lines suppressed due to ratelimiting [ 12.480878] SELinux: Disabled at runtime. [ 12.538620] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.548357] systemd[1]: Detected virtualization kvm. [ 12.550807] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.114125] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.118305] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.126906] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.131269] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.134989] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.144522] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.154434] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting POSIX Message Queue File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Switch Root. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... Mounting Huge Pages File System... [ OK ] Listening on udev Control Socket. [ 13.255034] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 13.609106] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 14.002736] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.008513] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.152868] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.163597] EDAC sbridge: Ver: 1.1.2 [ 15.631413] Key type dns_resolver registered [ 15.952472] NFS: Registering the id_resolver key type [ 15.954339] Key type id_resolver registered [ 15.958172] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg406-server login: [ 48.824613] libcfs: loading out-of-tree module taints kernel. [ 48.893258] Key type ._llcrypt registered [ 48.896338] Key type .llcrypt registered [ 49.046713] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 67.939349] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 69.640167] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 69.652414] alg: No test for adler32 (adler32-zlib) [ 71.208299] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 71.996031] hrtimer: interrupt took 2928178 ns [ 72.320066] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 74.129084] Key type lgssc registered [ 76.058071] Lustre: Echo OBD driver; http://www.lustre.org/ [ 91.592849] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 134.914678] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 147.975592] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 148.033398] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 149.282792] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 149.350753] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 149.451674] Lustre: lustre-MDT0000: new disk, initializing [ 149.603667] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 149.636477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 154.197052] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 168.100334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 168.258424] Lustre: 6522:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 168.289363] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 168.296978] Lustre: Skipped 1 previous similar message [ 168.372716] Lustre: lustre-MDT0001: new disk, initializing [ 168.436521] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 168.463644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 168.480974] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 174.180115] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 179.931829] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 189.299137] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 189.560469] Lustre: lustre-OST0000: new disk, initializing [ 189.565346] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 189.574864] Lustre: 8461:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 189.632589] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.967752] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.687225] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 196.707110] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 196.842509] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 214.049396] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 214.142325] Lustre: lustre-OST0001: new disk, initializing [ 214.146783] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 214.155676] Lustre: 9535:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 214.222950] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 220.281505] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 220.734356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 220.741266] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 220.803752] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 232.297497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 238.056889] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 245.008518] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing check_logdir /tmp/testlogs/ [ 252.292736] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing yml_node [ 257.011630] Lustre: DEBUG MARKER: Client: 2.17.54.83 [ 260.737924] Lustre: DEBUG MARKER: MDS: 2.17.54.83 [ 263.628882] Lustre: DEBUG MARKER: OSS: 2.17.54.83 [ 265.756111] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Jun 23 09:15:31 EDT 2026 [ 282.768973] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 284.173772] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 301.539144] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 301.550864] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 301.573608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 302.566076] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 302.567279] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 302.581199] Lustre: Skipped 1 previous similar message [ 305.650659] Lustre: server umount lustre-MDT0000 complete [ 309.197306] LustreError: 6514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782220576 with bad export cookie 496368384291286530 [ 309.201616] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 309.216087] LustreError: 6514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 312.800918] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 312.802041] LustreError: 6527:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 312.806515] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 312.806525] Lustre: Skipped 3 previous similar messages [ 312.809180] Lustre: Skipped 2 previous similar messages [ 312.839545] LustreError: 6527:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 314.849943] LustreError: 8468:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 314.849982] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 314.874028] LustreError: 8468:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 314.895847] Lustre: Skipped 2 previous similar messages [ 315.668510] Lustre: server umount lustre-MDT0001 complete [ 319.884683] Lustre: server umount lustre-OST0000 complete [ 323.830111] Lustre: server umount lustre-OST0001 complete [ 330.163428] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 338.122554] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 374.148313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 374.495225] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 374.538487] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 374.601238] Lustre: lustre-MDT0000: new disk, initializing [ 374.656382] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 374.677745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 379.532535] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 390.409439] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 390.623715] Lustre: 15082:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 390.751634] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 390.759180] Lustre: Skipped 1 previous similar message [ 390.889490] Lustre: lustre-MDT0001: new disk, initializing [ 391.019479] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 391.069530] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 391.076302] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 396.543753] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 406.649575] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 412.711942] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 419.527961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 419.786323] Lustre: lustre-OST0000: new disk, initializing [ 419.790884] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 419.808571] Lustre: 17054:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 419.943178] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 421.617616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 421.632395] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 421.761154] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 426.286057] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 437.567576] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 444.067489] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 444.420970] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 450.216390] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 450.449788] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 457.703096] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 457.709747] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 457.733438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 457.740541] Lustre: Skipped 1 previous similar message [ 459.386179] Lustre: server umount lustre-OST0000 complete [ 462.822611] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 462.825893] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 462.827886] LustreError: 3654:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff96e8754dfb80 x1868793475384448/t0(0) o8->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 462.873030] Lustre: Skipped 2 previous similar messages [ 467.938793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 467.960090] Lustre: Skipped 3 previous similar messages [ 477.152546] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 477.388308] Lustre: server umount lustre-MDT0000 complete [ 478.180094] LustreError: 15092:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 478.206681] LustreError: 15092:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 481.281597] LustreError: 15075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782220748 with bad export cookie 496368384291288063 [ 481.285525] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 481.304780] LustreError: 15075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 481.661484] Lustre: server umount lustre-MDT0001 complete [ 486.650345] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:19:13 (1782220753) [ 493.210596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 493.643924] LustreError: 19320:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 493.712316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 497.919256] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 499.175537] LustreError: 19321:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 505.747339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 510.144623] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 517.221078] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 523.138261] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 529.794557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 536.480762] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 543.799957] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 547.333659] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 553.955565] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 553.964571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 553.971723] Lustre: Skipped 2 previous similar messages [ 553.980191] Lustre: Skipped 4 previous similar messages [ 558.822682] Lustre: server umount lustre-MDT0000 complete [ 559.078766] LustreError: 21213:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 559.101498] LustreError: 21213:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 562.602072] LustreError: 19301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782220830 with bad export cookie 496368384291288952 [ 562.605154] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 562.616566] LustreError: 19301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 562.984633] Lustre: server umount lustre-MDT0001 complete [ 577.740932] Lustre: server umount lustre-OST0000 complete [ 599.962744] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 612.806921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 613.379063] LustreError: 23702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 613.484353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 613.492219] Lustre: Skipped 2 previous similar messages [ 618.589409] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 627.485699] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 632.647878] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 635.889405] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 641.396405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 647.739908] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 656.893951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 657.098357] Lustre: lustre-OST0001: new disk, initializing [ 657.102753] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 657.114833] Lustre: 26020:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 658.808062] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 658.820457] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 658.940289] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 663.753203] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 672.790367] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 675.986508] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 689.122329] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 689.131722] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 689.151510] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 692.010345] Lustre: server umount lustre-MDT0000 complete [ 695.443788] LustreError: 24841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782220963 with bad export cookie 496368384291289799 [ 695.444569] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 695.452078] LustreError: 24841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 695.738601] Lustre: server umount lustre-MDT0001 complete [ 710.077750] Lustre: server umount lustre-OST0000 complete [ 712.058012] Lustre: 3655:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1782220963/real 1782220963] req@ffff96e8445f1c00 x1868793475483392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1782220979 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 714.295481] Lustre: server umount lustre-OST0001 complete [ 731.418830] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 740.439437] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 740.826310] LustreError: 29039:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 740.839994] LustreError: 29039:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 745.091704] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 753.498805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 754.064505] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 754.075908] Lustre: Skipped 4 previous similar messages [ 757.948901] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 760.877175] Lustre: 30188:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 767.366542] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 774.182783] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 782.750694] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 789.733580] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 797.024301] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 800.512603] Lustre: 32066:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 806.887275] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 806.894808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 806.910566] Lustre: Skipped 7 previous similar messages [ 806.923224] Lustre: Skipped 6 previous similar messages [ 812.295684] Lustre: server umount lustre-MDT0000 complete [ 815.974062] LustreError: 29021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782221083 with bad export cookie 496368384291291094 [ 815.977310] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 815.990156] LustreError: 29021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 816.319589] Lustre: server umount lustre-MDT0001 complete [ 830.328458] Lustre: server umount lustre-OST0000 complete [ 832.482713] Lustre: 3657:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1782221084/real 1782221084] req@ffff96e741daad80 x1868793475532544/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1782221100 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 834.046781] Lustre: server umount lustre-OST0001 complete [ 842.542451] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:25:09 (1782221109) [ 869.539212] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 879.329680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 879.686502] LustreError: 34866:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 879.699198] LustreError: 34866:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 884.025538] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 892.018570] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 896.644053] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 899.564441] Lustre: 36013:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 906.019395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 911.704478] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 919.839513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 927.420343] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 937.088880] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 940.814586] Lustre: 37895:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 948.706698] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 948.716502] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 948.742183] Lustre: Skipped 1 previous similar message [ 948.748082] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 948.751635] Lustre: Skipped 5 previous similar messages [ 951.975369] Lustre: server umount lustre-MDT0000 complete [ 955.607240] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782221223 with bad export cookie 496368384291292200 [ 955.610211] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 955.629311] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 956.107748] Lustre: server umount lustre-MDT0001 complete [ 969.785873] Lustre: server umount lustre-OST0000 complete [ 983.756481] Lustre: server umount lustre-OST0001 complete [ 992.175701] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:27:38 (1782221258) [ 1008.026211] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1017.761914] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1018.226308] LustreError: 40039:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1018.256080] LustreError: 40039:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 1018.371109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1018.379916] Lustre: Skipped 6 previous similar messages [ 1022.433413] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1029.851787] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1034.708264] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1037.655227] Lustre: 41190:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1044.347808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1050.777748] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1058.663828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1064.916031] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1071.981623] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1075.321109] Lustre: 43067:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1076.115306] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1076.122048] Lustre: Skipped 1 previous similar message [ 1081.313137] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1081.325467] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1081.341113] Lustre: Skipped 3 previous similar messages [ 1085.148449] Lustre: server umount lustre-MDT0000 complete [ 1088.076355] LustreError: 40020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782221355 with bad export cookie 496368384291293306 [ 1088.080928] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1088.082894] LustreError: 40020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1094.446966] Lustre: server umount lustre-MDT0001 complete [ 1101.500584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1117.072834] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1123.249033] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1124.914258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1129.951293] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1130.486320] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1130.496502] Lustre: Skipped 1 previous similar message [ 1137.885053] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1139.415585] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1144.946753] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1146.716790] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1156.080451] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1158.020391] Lustre: server umount lustre-MDT0000 complete [ 1161.605104] LustreError: 41192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782221429 with bad export cookie 496368384291294433 [ 1161.607971] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1161.617031] LustreError: 41192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1162.086597] Lustre: server umount lustre-MDT0001 complete [ 1175.614243] Lustre: server umount lustre-OST0000 complete [ 1189.248754] Lustre: server umount lustre-OST0001 complete [ 1197.719691] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:31:04 (1782221464) [ 1215.331484] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1227.494659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1232.501343] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1241.545946] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1247.702592] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1251.034758] Lustre: 48825:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1258.209896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1265.374817] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1274.327210] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1281.280172] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1288.179944] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1292.502829] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1304.817069] Lustre: server umount lustre-MDT0000 complete [ 1305.576690] LustreError: 47670:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1305.588789] LustreError: 47670:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 28 previous similar messages [ 1308.258613] LustreError: 47655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782221575 with bad export cookie 496368384291295231 [ 1308.262517] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1308.269439] LustreError: 47655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1308.652930] Lustre: server umount lustre-MDT0001 complete [ 1322.305452] Lustre: server umount lustre-OST0000 complete [ 1336.135407] Lustre: server umount lustre-OST0001 complete [ 1344.841137] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:33:31 (1782221611) [ 1347.152695] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1348.837976] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:33:35 (1782221615) [ 1359.473239] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1368.251800] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1389.839446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1390.058806] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1390.147713] Lustre: lustre-MDT0000: new disk, initializing [ 1390.265968] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1394.248781] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1403.770953] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1410.318030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1410.532342] Lustre: lustre-OST0000: new disk, initializing [ 1410.534753] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1410.537445] Lustre: Skipped 1 previous similar message [ 1410.540670] Lustre: 54903:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1412.293498] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1412.303247] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1412.326858] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1417.828918] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1426.804831] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1428.930791] Lustre: DEBUG MARKER: create test files [ 1463.781210] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1463.790458] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1463.794946] Lustre: Skipped 13 previous similar messages [ 1463.796872] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1463.796880] Lustre: Skipped 19 previous similar messages [ 1465.277633] Lustre: server umount lustre-OST0000 complete [ 1469.330383] Lustre: server umount lustre-MDT0000 complete [ 1480.540237] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 1483.434380] Key type lgssc unregistered [ 1483.847496] LNet: 56533:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1483.868657] LNetError: 56533:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1483.894145] LNet: Removed LNI 192.168.204.106@tcp [ 1484.986183] Key type .llcrypt unregistered [ 1484.989220] Key type ._llcrypt unregistered [ 1512.768964] Key type ._llcrypt registered [ 1512.770894] Key type .llcrypt registered [ 1512.875554] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1513.861421] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1513.969141] alg: No test for adler32 (adler32-zlib) [ 1515.075623] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 1515.337409] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 1517.009138] Key type lgssc registered [ 1518.032585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1524.419890] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1524.441649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1525.863703] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1530.234383] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1537.148925] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1542.938978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1543.279917] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1549.149784] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1550.441888] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1550.546160] Lustre: 59267:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1550.553667] Lustre: 59267:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1550.568293] Lustre: 59267:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1550.582303] Lustre: 59267:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1550.588070] Lustre: 59267:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1550.596729] Lustre: 59267:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1556.696487] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1562.785627] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1567.930990] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1576.155736] Lustre: server umount lustre-OST0000 complete [ 1590.462984] Lustre: server umount lustre-MDT0000 complete [ 1601.068215] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 1604.014782] Key type lgssc unregistered [ 1604.320303] LNet: 60877:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1604.336948] LNetError: 60877:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1604.355627] LNet: Removed LNI 192.168.204.106@tcp [ 1605.222073] Key type .llcrypt unregistered [ 1605.224898] Key type ._llcrypt unregistered [ 1633.662337] Key type ._llcrypt registered [ 1633.666295] Key type .llcrypt registered [ 1633.769103] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1635.083274] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1635.147510] alg: No test for adler32 (adler32-zlib) [ 1636.230601] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 1636.478245] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 1638.160385] Key type lgssc registered [ 1639.376199] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1647.110808] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1647.127420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1648.658776] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1653.898680] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1660.473043] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1667.088401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1667.456777] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1672.933621] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1673.597564] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1673.672122] Lustre: 63703:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1673.689223] Lustre: 63703:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1673.696306] Lustre: 63703:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1673.701178] Lustre: 63703:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1673.714637] Lustre: 63703:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1673.736890] Lustre: 63703:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1678.955834] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1683.371794] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1694.182618] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1694.184307] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1694.213354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1696.778440] Lustre: server umount lustre-OST0000 complete [ 1700.161055] Lustre: server umount lustre-MDT0000 complete [ 1709.349355] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 1712.318811] Key type lgssc unregistered [ 1712.648039] LNet: 65262:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1712.657796] LNetError: 65262:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1712.683153] LNet: Removed LNI 192.168.204.106@tcp [ 1713.614295] Key type .llcrypt unregistered [ 1713.624966] Key type ._llcrypt unregistered [ 1730.350386] Key type ._llcrypt registered [ 1730.355149] Key type .llcrypt registered [ 1730.546438] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1746.518853] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1747.248296] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1747.271311] alg: No test for adler32 (adler32-zlib) [ 1748.447756] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 1748.675193] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 1750.402077] Key type lgssc registered [ 1751.660576] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1785.325639] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1785.341969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1786.559524] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1786.601074] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1786.662327] Lustre: lustre-MDT0000: new disk, initializing [ 1786.767294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1786.801558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1790.409209] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1799.605440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1799.713895] Lustre: 68414:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1799.766676] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1799.771220] Lustre: Skipped 1 previous similar message [ 1799.867802] Lustre: lustre-MDT0001: new disk, initializing [ 1799.962871] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1800.015269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1800.026254] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1803.816287] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1810.982417] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1815.486438] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1820.872812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1821.138878] Lustre: lustre-OST0000: new disk, initializing [ 1821.145987] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1821.153701] Lustre: 70383:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1821.270598] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1826.907583] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1829.956114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1829.969566] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1830.121254] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1835.971412] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1840.806535] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1841.004353] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1845.291782] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1845.460740] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1850.854612] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1850.856359] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1850.886149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1850.895566] Lustre: Skipped 1 previous similar message [ 1853.362297] Lustre: server umount lustre-OST0000 complete [ 1861.092064] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1861.108077] Lustre: Skipped 1 previous similar message [ 1861.120172] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1861.125746] Lustre: Skipped 1 previous similar message [ 1863.632491] Lustre: server umount lustre-MDT0000 complete [ 1866.777252] LustreError: 68406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782222134 with bad export cookie 3970528926639712717 [ 1866.782970] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1866.789978] LustreError: 68406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1867.140373] Lustre: server umount lustre-MDT0001 complete [ 1874.454705] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:42:20 (1782222140) [ 1884.895761] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1921.848458] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1930.911749] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1966.395657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1966.794427] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1966.844505] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1966.974634] Lustre: lustre-MDT0000: new disk, initializing [ 1967.136337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1967.171095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1971.597903] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1980.952207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1981.083814] Lustre: 76400:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1981.141436] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1981.144456] Lustre: Skipped 1 previous similar message [ 1981.208918] Lustre: lustre-MDT0001: new disk, initializing [ 1981.319740] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1981.370942] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1981.383896] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1985.822603] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1994.620217] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2000.176725] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2006.293760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2006.569291] Lustre: lustre-OST0000: new disk, initializing [ 2006.573710] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2006.581220] Lustre: 78370:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2006.678167] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2013.299791] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2016.295480] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2016.303632] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2016.399951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2022.846249] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2027.145544] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2027.320194] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2031.353875] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2031.578966] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2036.709241] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2036.716170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2036.732226] Lustre: Skipped 2 previous similar messages [ 2036.740232] Lustre: Skipped 1 previous similar message [ 2039.344388] Lustre: server umount lustre-OST0000 complete [ 2042.853665] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2042.859025] LustreError: Skipped 1 previous similar message [ 2042.866441] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2042.883858] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2046.963231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2046.964555] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2048.826789] Lustre: server umount lustre-MDT0000 complete [ 2052.070039] LustreError: 77533:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2052.097493] LustreError: 77533:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2052.182728] LustreError: 79710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782222319 with bad export cookie 3970528926639713599 [ 2052.188539] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2052.195379] LustreError: 79710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2052.589790] Lustre: server umount lustre-MDT0001 complete [ 2060.488962] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:45:27 (1782222327) [ 2067.119392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2067.736273] LustreError: 80782:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2067.815899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2072.212092] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2073.060105] LustreError: 80783:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2078.181778] LustreError: 80782:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2079.964921] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2084.142209] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2091.359512] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2095.525171] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2102.484962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2102.857667] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2102.871833] Lustre: Skipped 1 previous similar message [ 2110.229915] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2119.266962] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2123.043657] Lustre: 83454:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 3b3b7a70-ac62-4ea2-8dcd-424fe076d410 at adminstrative request [ 2123.753242] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2123.765875] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2133.999587] Lustre: MGS: Client 3b3b7a70-ac62-4ea2-8dcd-424fe076d410 (at 0@lo) reconnecting [ 2134.013288] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x371a2705a2c5f0b8 to 0x371a2705a2c5f358 [ 2134.030567] Lustre: MGC192.168.204.106@tcp: Connection restored to 0@lo (at 0@lo) [ 2159.814028] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2159.994725] Lustre: lustre-OST0001: new disk, initializing [ 2160.000124] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2160.013718] Lustre: 83847:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2167.026841] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2167.953576] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2167.961938] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2168.128991] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2176.887159] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2183.650228] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2183.657435] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2183.670103] LustreError: Skipped 1 previous similar message [ 2183.683235] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2183.691640] Lustre: Skipped 2 previous similar messages [ 2186.579845] Lustre: server umount lustre-OST0001 complete [ 2188.775762] LustreError: 82602:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2188.800064] LustreError: 82602:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2190.306129] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2199.012752] LustreError: 82604:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2199.028206] LustreError: 82604:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2204.129593] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2204.141355] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2204.146411] Lustre: Skipped 7 previous similar messages [ 2204.268510] Lustre: server umount lustre-OST0000 complete [ 2208.230360] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2208.241341] LustreError: Skipped 1 previous similar message [ 2208.247740] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2208.264014] Lustre: Skipped 3 previous similar messages [ 2222.048169] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2222.377335] Lustre: server umount lustre-MDT0000 complete [ 2224.609239] LustreError: 80779:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2224.621233] LustreError: 80779:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2225.927094] LustreError: 80762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782222493 with bad export cookie 3970528926639715160 [ 2225.932105] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2225.939370] LustreError: 80762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2226.215078] Lustre: server umount lustre-MDT0001 complete [ 2234.199269] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:48:20 (1782222500) [ 2235.811976] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2237.841430] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:48:23 (1782222503) [ 2239.566431] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2241.339151] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:48:27 (1782222507) [ 2242.872230] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2244.470834] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:48:30 (1782222510) [ 2245.830692] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2247.367176] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:48:33 (1782222513) [ 2248.912673] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2256.008652] loop: module loaded [ 2256.024817] loop0: detected capacity change from 0 to 4294967296 [ 2256.111095] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2258.132389] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2261.299208] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2265.003947] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2291.107160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2291.141399] LustreError: 87973:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2291.147804] LustreError: 87973:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2291.152901] LustreError: 87973:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2291.161926] LustreError: 87973:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2308.826363] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 2318.664800] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2353.751249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2354.076426] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2354.127555] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2354.215778] Lustre: lustre-MDT0000: new disk, initializing [ 2354.323137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2354.331135] Lustre: Skipped 1 previous similar message [ 2354.355438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2359.722973] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2370.607863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2370.747429] Lustre: 90837:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2370.988719] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2375.418516] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2385.034205] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2391.230648] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2398.828246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2399.150707] Lustre: 92807:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2405.817267] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2407.578386] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2407.800429] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2415.727961] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2420.747761] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2421.081522] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2426.462975] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2426.810975] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2433.505976] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2433.506775] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2433.519549] LustreError: Skipped 1 previous similar message [ 2433.529400] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2433.547277] Lustre: Skipped 2 previous similar messages [ 2433.573871] Lustre: Skipped 8 previous similar messages [ 2435.912223] Lustre: server umount lustre-OST0000 complete [ 2448.309794] Lustre: server umount lustre-MDT0000 complete [ 2448.868167] LustreError: 90844:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2448.883751] LustreError: 90844:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2451.989522] LustreError: 90831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782222719 with bad export cookie 3970528926639715776 [ 2451.992905] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2451.995102] LustreError: 90831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2452.219732] Lustre: server umount lustre-MDT0001 complete [ 2460.761794] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:52:06 (1782222726) [ 2468.151879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2473.742490] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2482.787303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2483.269208] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2483.272430] Lustre: Skipped 3 previous similar messages [ 2487.977162] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2494.823914] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2501.361410] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2508.921281] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2516.103120] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2524.985797] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2529.397964] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2529.638334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2533.825858] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2534.033079] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2540.230867] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2540.386272] Lustre: lustre-OST0001: new disk, initializing [ 2540.389713] Lustre: Skipped 2 previous similar messages [ 2540.394403] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2540.401115] Lustre: Skipped 3 previous similar messages [ 2540.405861] Lustre: 98593:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2546.946688] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2549.823402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2549.830547] Lustre: Skipped 2 previous similar messages [ 2549.840739] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2549.943147] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2556.124956] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2560.737840] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2560.938084] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2565.621575] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2565.864853] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2584.576899] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2608.032663] Lustre: Failing over lustre-MDT0000 [ 2608.456519] Lustre: server umount lustre-MDT0000 complete [ 2609.121556] LustreError: 99993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2609.159328] LustreError: 99993:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2611.169423] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2611.174435] LustreError: Skipped 1 previous similar message [ 2611.179566] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2611.196406] Lustre: Skipped 2 previous similar messages [ 2628.065398] Lustre: 66637:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1782222879/real 1782222879] req@00000000a7443465 x1868795232920320/t0(0) o400->MGC192.168.204.106@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1782222895 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2628.107213] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2632.910760] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2632.923519] LDISKFS-fs (dm-0): recovery complete [ 2632.955214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2638.324364] LustreError: 66635:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000003b919825 x1868795232929280/t0(0) o250->MGC192.168.204.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2638.648459] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2640.829761] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2640.836532] Lustre: 96465:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2643.619382] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2643.946069] Lustre: 95223:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2643.971656] Lustre: 95223:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2644.005440] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2644.083536] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2644.417612] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2644.767211] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2644.786223] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2645.112211] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2645.118203] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2645.126994] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2645.146450] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2645.801028] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2645.813147] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2646.139821] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2646.156737] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2647.424414] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2647.442182] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 2648.083634] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2648.100431] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2648.424218] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2648.433271] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2651.585406] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2651.595340] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2652.233110] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2652.253112] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2652.585439] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2652.592352] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2659.628956] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2659.646272] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 24 previous similar messages [ 2660.301456] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2660.312559] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2660.632314] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2660.639267] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2675.650392] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2675.661552] Lustre: 101441:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 49 previous similar messages [ 2676.307654] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2676.316739] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 49 previous similar messages [ 2676.640223] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2676.644433] LustreError: 101441:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 49 previous similar messages [ 2700.216203] Lustre: 101441:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2700.223903] Lustre: 101441:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2700.233205] Lustre: 101441:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 5829ead6-142a-4f1e-85e7-f6906651ac23@192.168.204.6@tcp [ 2700.246319] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2700.255538] Lustre: 101441:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@00000000421ea660 x1868795216988672/t0(8589934772) o35->5829ead6-142a-4f1e-85e7-f6906651ac23@192.168.204.6@tcp:178/0 lens 392/0 e 0 to 0 dl 1782222978 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2700.305199] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2700.315357] Lustre: lustre-MDT0000: Denying connection for new client 5829ead6-142a-4f1e-85e7-f6906651ac23 (at 192.168.204.6@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2700.362369] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2700.435980] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2700.441984] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2700.456585] Lustre: Skipped 2 previous similar messages [ 2700.529788] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2700.533837] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2715.633335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2715.650863] Lustre: Skipped 4 previous similar messages [ 2720.367675] Lustre: server umount lustre-OST0000 complete [ 2730.595340] Lustre: server umount lustre-OST0001 complete [ 2740.964914] Lustre: server umount lustre-MDT0000 complete [ 2741.226372] LustreError: 95223:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2741.243895] LustreError: 95223:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 46 previous similar messages [ 2744.658814] LustreError: 95203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782223012 with bad export cookie 3970528926639766995 [ 2744.662528] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2744.674557] LustreError: 95203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2745.206826] Lustre: server umount lustre-MDT0001 complete [ 2752.462667] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:56:58 (1782223018) [ 2759.377679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2759.384594] Lustre: *** cfs_fail_loc=197, val=0*** [ 2775.072633] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2780.193146] LustreError: 103415:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.106@tcp: failed processing log, type 4: rc = -110 [ 2785.418430] Lustre: *** cfs_fail_loc=197, val=0*** [ 2785.422469] Lustre: Skipped 65 previous similar messages [ 2805.984392] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2805.987681] Lustre: Skipped 3 previous similar messages [ 2811.050277] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2818.699720] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2821.016188] Lustre: server umount lustre-OST0000 complete [ 2829.141248] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:58:15 (1782223095) [ 2849.051702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2853.769366] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2862.983595] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2866.877283] LustreError: 105073:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2866.891257] LustreError: 105073:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2866.904583] LustreError: 105073:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 2867.187647] Lustre: server umount lustre-MDT0000 complete [ 2894.765572] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 2904.879940] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2939.612258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2939.857348] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2939.885415] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2939.989874] Lustre: lustre-MDT0000: new disk, initializing [ 2940.093704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2943.731529] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2953.604203] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2953.733634] Lustre: 109245:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2954.000989] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2958.174623] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2966.880902] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2972.327824] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2978.200578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2978.499582] Lustre: 111211:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2983.728161] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2988.756029] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2992.913735] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2998.411527] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2998.581463] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3004.022664] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3004.263387] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3009.513023] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3009.514013] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3009.530674] Lustre: Skipped 9 previous similar messages [ 3009.547046] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3009.550610] LustreError: Skipped 1 previous similar message [ 3009.572490] Lustre: Skipped 5 previous similar messages [ 3012.406427] Lustre: server umount lustre-OST0000 complete [ 3023.054282] Lustre: server umount lustre-MDT0000 complete [ 3024.866637] LustreError: 109256:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3024.888233] LustreError: 109256:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3026.112884] LustreError: 109236:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782223293 with bad export cookie 3970528926639777607 [ 3026.114258] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3026.122655] LustreError: 109236:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3026.355517] Lustre: server umount lustre-MDT0001 complete [ 3033.360626] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:01:40 (1782223300) [ 3041.064388] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3043.642900] Key type lgssc unregistered [ 3043.881566] LNet: 113711:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3043.893650] LNetError: 113711:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3043.922369] LNet: Removed LNI 192.168.204.106@tcp [ 3044.729573] Key type .llcrypt unregistered [ 3044.732907] Key type ._llcrypt unregistered [ 3061.079171] Key type ._llcrypt registered [ 3061.080615] Key type .llcrypt registered [ 3061.205456] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3076.447977] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3077.662759] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3077.715644] alg: No test for adler32 (adler32-zlib) [ 3078.734425] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 3079.035826] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 3080.680874] Key type lgssc registered [ 3081.864582] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3122.292689] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3122.310553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3123.692148] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3123.751387] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3123.868433] Lustre: lustre-MDT0000: new disk, initializing [ 3123.974935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3123.995268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3128.551836] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3138.863770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3139.115737] Lustre: lustre-OST0000: new disk, initializing [ 3139.123215] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3139.134740] Lustre: Skipped 1 previous similar message [ 3139.153274] Lustre: 117885:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3139.224309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3140.518462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3140.527913] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3140.609701] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3146.088301] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3156.418221] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3156.612747] Lustre: lustre-OST0001: new disk, initializing [ 3156.616890] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3156.630448] Lustre: 118748:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3156.718689] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3162.137570] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3163.225034] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3163.234039] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3163.275294] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3173.357383] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3173.359842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3173.378988] Lustre: Skipped 1 previous similar message [ 3176.208086] Lustre: server umount lustre-MDT0000 complete [ 3182.759487] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3187.565740] LustreError: 117072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782223455 with bad export cookie 13269962630246977306 [ 3187.585727] LustreError: 117072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3187.592054] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3187.712230] Lustre: server umount lustre-OST0000 complete [ 3192.073854] Lustre: server umount lustre-OST0001 complete [ 3199.896955] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 10:04:26 (1782223466) [ 3210.106965] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3231.523495] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:04:57 (1782223497) [ 3268.024713] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3283.078489] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3323.439288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3323.690926] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3323.718931] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3323.855466] Lustre: lustre-MDT0000: new disk, initializing [ 3323.956094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3323.972413] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3328.918465] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3339.556969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3339.764898] Lustre: 125509:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3339.850353] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3339.859910] Lustre: Skipped 1 previous similar message [ 3339.935191] Lustre: lustre-MDT0001: new disk, initializing [ 3340.055808] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3340.099573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3340.107512] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3345.263713] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3354.822562] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3361.014814] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3367.931277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3368.216745] Lustre: lustre-OST0000: new disk, initializing [ 3368.224537] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3368.235231] Lustre: 127478:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3368.371766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3375.310718] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3377.231171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3377.251382] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3377.399275] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3386.085322] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3391.897736] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3392.201029] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3398.004901] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3398.288560] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3403.260126] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3403.264126] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3403.290686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3403.303061] LustreError: Skipped 1 previous similar message [ 3403.325503] Lustre: Skipped 2 previous similar messages [ 3406.927101] Lustre: server umount lustre-OST0000 complete [ 3411.945434] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3411.957333] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3411.976842] Lustre: Skipped 1 previous similar message [ 3411.980716] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3418.595878] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3418.606072] Lustre: Skipped 3 previous similar messages [ 3423.713110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3423.724770] Lustre: Skipped 1 previous similar message [ 3425.248488] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3425.679510] Lustre: server umount lustre-MDT0000 complete [ 3429.972782] LustreError: 125500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782223697 with bad export cookie 13269962630246978524 [ 3429.973845] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3429.991406] LustreError: 125500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3430.442747] Lustre: server umount lustre-MDT0001 complete [ 3440.014184] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:08:26 (1782223706) [ 3446.791765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3447.218096] LustreError: 129892:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3447.238460] LustreError: 129892:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3447.289570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3451.406199] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3452.398609] LustreError: 129893:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3456.494603] LustreError: 129892:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3459.618679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3464.676352] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3472.654339] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3478.412254] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3484.612669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3484.980653] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3484.989086] Lustre: Skipped 1 previous similar message [ 3490.698830] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3497.926454] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3502.312028] Lustre: *** cfs_fail_loc=159, val=0*** [ 3519.023866] Lustre: lustre-MDT0000: Client b4593016-a3b4-4311-a21c-72d7e6816888 (at 192.168.204.6@tcp) reconnecting [ 3520.611410] Lustre: *** cfs_fail_loc=159, val=0*** [ 3520.619310] Lustre: Skipped 6 previous similar messages [ 3536.932296] Lustre: lustre-MDT0000: Client b4593016-a3b4-4311-a21c-72d7e6816888 (at 192.168.204.6@tcp) reconnecting [ 3542.507500] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3542.514071] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3542.533232] Lustre: Skipped 1 previous similar message [ 3542.538242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3548.498109] Lustre: server umount lustre-OST0000 complete [ 3553.774155] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3553.787859] Lustre: Skipped 1 previous similar message [ 3558.688983] Lustre: server umount lustre-MDT0000 complete [ 3558.883156] LustreError: 129893:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3558.897408] LustreError: 129893:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3562.147548] LustreError: 129872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782223829 with bad export cookie 13269962630246979413 [ 3562.151230] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3562.157903] LustreError: 129872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3562.633845] Lustre: server umount lustre-MDT0001 complete [ 3573.425061] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3576.409812] Key type lgssc unregistered [ 3576.793220] LNet: 133662:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3576.816511] LNetError: 133662:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3577.905527] LNet: Removed LNI 192.168.204.106@tcp [ 3578.784243] Key type .llcrypt unregistered [ 3578.786883] Key type ._llcrypt unregistered [ 3605.333164] Key type ._llcrypt registered [ 3605.338329] Key type .llcrypt registered [ 3605.449711] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3606.764269] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3606.794272] alg: No test for adler32 (adler32-zlib) [ 3607.903091] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 3608.185339] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 3609.856237] Key type lgssc registered [ 3611.193839] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3620.747102] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3620.781334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3622.341402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3627.788628] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3632.611533] LustreError: 135078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3632.632815] LustreError: 135078:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3637.401939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3637.866738] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3642.803117] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3652.026536] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3657.963601] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3666.056227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3666.609568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3667.648666] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3673.856088] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3682.853626] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3693.543691] Lustre: *** cfs_fail_loc=159, val=0*** [ 3709.945236] Lustre: lustre-MDT0000: Client d2f86fe2-1e42-473f-bab7-4b504dab7840 (at 192.168.204.6@tcp) reconnecting [ 3712.849149] Lustre: *** cfs_fail_loc=159, val=0*** [ 3728.399738] Lustre: lustre-MDT0001: Client d2f86fe2-1e42-473f-bab7-4b504dab7840 (at 192.168.204.6@tcp) reconnecting [ 3730.030941] Lustre: *** cfs_fail_loc=159, val=0*** [ 3730.041246] Lustre: Skipped 4 previous similar messages [ 3745.303138] Lustre: lustre-MDT0001: Client d2f86fe2-1e42-473f-bab7-4b504dab7840 (at 192.168.204.6@tcp) reconnecting [ 3750.804209] Lustre: 138113:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3754.118935] Lustre: *** cfs_fail_loc=159, val=0*** [ 3754.120794] Lustre: Skipped 3 previous similar messages [ 3769.365802] Lustre: lustre-MDT0000: Client f09625da-11a4-4e3d-aeb5-9ff7b2013cb9 (at 192.168.204.6@tcp) reconnecting [ 3771.457362] Lustre: *** cfs_fail_loc=159, val=0*** [ 3771.462748] Lustre: Skipped 14 previous similar messages [ 3787.286975] Lustre: lustre-MDT0000: Client f09625da-11a4-4e3d-aeb5-9ff7b2013cb9 (at 192.168.204.6@tcp) reconnecting [ 3789.134436] Lustre: 138417:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3794.921479] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3794.933158] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3794.938219] LustreError: Skipped 1 previous similar message [ 3794.970764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3795.937150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3795.943629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3795.965560] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3795.971948] LustreError: Skipped 4 previous similar messages [ 3800.753677] Lustre: server umount lustre-OST0000 complete [ 3806.177514] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3806.194183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3806.207866] Lustre: Skipped 2 previous similar messages [ 3811.098028] Lustre: server umount lustre-MDT0000 complete [ 3811.300070] LustreError: 135077:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3811.326943] LustreError: 135077:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3815.387416] LustreError: 138367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782224083 with bad export cookie 5708904573870547192 [ 3815.388871] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3815.402839] LustreError: 138367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3815.739404] Lustre: server umount lustre-MDT0001 complete [ 3828.144605] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3831.072720] Key type lgssc unregistered [ 3831.347608] LNet: 139471:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3831.351769] LNetError: 139471:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3831.371722] LNet: Removed LNI 192.168.204.106@tcp [ 3832.321370] Key type .llcrypt unregistered [ 3832.323477] Key type ._llcrypt unregistered [ 3858.907888] Key type ._llcrypt registered [ 3858.909325] Key type .llcrypt registered [ 3859.084910] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3860.312063] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3860.372083] alg: No test for adler32 (adler32-zlib) [ 3861.487211] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 3861.787161] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 3863.528165] Key type lgssc registered [ 3864.902965] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3873.627365] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3873.658482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3875.301775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3879.566383] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3884.517037] LustreError: 140887:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3884.538843] LustreError: 140887:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3888.705901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3889.308863] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3895.365120] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3905.036573] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3910.926339] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3918.480466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3918.871916] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3919.949983] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3926.623082] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3934.278905] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3934.781255] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3941.585674] Lustre: 143471:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3944.618707] Lustre: 143519:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3945.882845] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3946.719464] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3950.530956] Lustre: 143668:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3951.470611] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3952.263325] Lustre: 143766:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3959.777108] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3959.784068] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3959.798637] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3959.803730] Lustre: Skipped 1 previous similar message [ 3962.425183] Lustre: server umount lustre-OST0000 complete [ 3970.023706] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3970.032618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3970.048362] Lustre: Skipped 1 previous similar message [ 3970.061416] Lustre: Skipped 2 previous similar messages [ 3972.576276] Lustre: server umount lustre-MDT0000 complete [ 3975.145611] LustreError: 141989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3975.170298] LustreError: 141989:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3977.469861] LustreError: 140867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782224245 with bad export cookie 1278582768870844651 [ 3977.490122] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3977.491477] LustreError: 140867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3978.412753] Lustre: server umount lustre-MDT0001 complete [ 3990.965721] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3994.419581] Key type lgssc unregistered [ 3994.709111] LNet: 144832:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3994.715457] LNetError: 144832:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3994.734135] LNet: Removed LNI 192.168.204.106@tcp [ 3995.716983] Key type .llcrypt unregistered [ 3995.718817] Key type ._llcrypt unregistered [ 4022.311563] Key type ._llcrypt registered [ 4022.313127] Key type .llcrypt registered [ 4022.447265] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4024.170930] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4024.260925] alg: No test for adler32 (adler32-zlib) [ 4025.490723] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 4026.009267] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4027.809756] Key type lgssc registered [ 4029.445222] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4038.492787] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4038.534274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4040.094128] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4044.819828] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4049.381374] LustreError: 146247:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4049.408110] LustreError: 146247:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4054.329562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4054.498566] LustreError: 146248:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4054.816161] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4059.705422] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4067.633991] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4073.675254] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4081.016152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4081.444975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4084.532622] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4088.403573] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4095.753297] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4099.886876] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4104.820149] Lustre: *** cfs_fail_loc=159, val=0*** [ 4104.830206] Lustre: Skipped 1 previous similar message [ 4121.099876] Lustre: lustre-MDT0000: Client fd72003d-1996-4962-9118-3bc7181f8262 (at 192.168.204.6@tcp) reconnecting [ 4130.792935] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4130.814627] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4130.817227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4133.752386] Lustre: server umount lustre-OST0000 complete [ 4141.027316] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4141.043386] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4141.060430] Lustre: Skipped 1 previous similar message [ 4141.062217] Lustre: Skipped 2 previous similar messages [ 4146.148228] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4146.157679] Lustre: Skipped 2 previous similar messages [ 4151.269534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4152.288225] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4152.625306] Lustre: server umount lustre-MDT0000 complete [ 4156.386035] LustreError: 146248:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4156.410138] LustreError: 146248:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4156.631476] LustreError: 146229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782224424 with bad export cookie 6541803762586492481 [ 4156.639834] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4156.648351] LustreError: 146229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4157.033582] Lustre: server umount lustre-MDT0001 complete [ 4169.154621] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4172.313579] Key type lgssc unregistered [ 4172.675201] LNet: 149914:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4172.694788] LNetError: 149914:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4173.733835] LNet: Removed LNI 192.168.204.106@tcp [ 4174.813630] Key type .llcrypt unregistered [ 4174.815379] Key type ._llcrypt unregistered [ 4203.347840] Key type ._llcrypt registered [ 4203.352141] Key type .llcrypt registered [ 4203.551335] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4204.803668] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4204.859093] alg: No test for adler32 (adler32-zlib) [ 4206.015507] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 4206.379975] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4208.128305] Key type lgssc registered [ 4209.307934] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4217.725922] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4217.746327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4219.270326] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4224.407398] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4228.580091] LustreError: 151333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4228.600394] LustreError: 151333:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4233.071455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4233.503665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4236.961290] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4244.163127] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4252.068739] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4260.803203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4261.097734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4264.258825] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4268.202291] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4275.593526] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4279.605877] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4299.636711] Lustre: lustre-MDT0001: evicting fddaae4f-854c-47cd-ae6b-dab0d395ba4f (at 192.168.204.6@tcp) by administrative request [ 4299.651225] Lustre: Skipped 1 previous similar message [ 4330.978962] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4330.979174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4330.985326] LustreError: Skipped 1 previous similar message [ 4330.992174] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4330.998223] Lustre: Skipped 1 previous similar message [ 4334.592992] Lustre: server umount lustre-OST0000 complete [ 4341.227373] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4341.239541] Lustre: Skipped 1 previous similar message [ 4341.247994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4341.250786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4341.259411] Lustre: Skipped 1 previous similar message [ 4344.627961] Lustre: server umount lustre-MDT0000 complete [ 4346.339186] LustreError: 151327:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4346.355625] LustreError: 151327:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4349.409915] LustreError: 151314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782224617 with bad export cookie 8463822846875889060 [ 4349.416094] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4349.421873] LustreError: 151314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4349.772785] Lustre: server umount lustre-MDT0001 complete [ 4362.463364] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4364.964389] Key type lgssc unregistered [ 4365.212639] LNet: 155401:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4365.219708] LNetError: 155401:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4365.240788] LNet: Removed LNI 192.168.204.106@tcp [ 4365.972844] Key type .llcrypt unregistered [ 4365.975698] Key type ._llcrypt unregistered [ 4440.514507] Key type ._llcrypt registered [ 4440.517497] Key type .llcrypt registered [ 4440.626312] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4441.909431] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4441.993184] alg: No test for adler32 (adler32-zlib) [ 4443.421407] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 4443.728341] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4445.464185] Key type lgssc registered [ 4446.864533] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4456.161573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4456.207733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4457.962171] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4463.412978] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4468.206420] LustreError: 157892:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4468.240575] LustreError: 157892:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4472.624761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4473.132797] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4478.100473] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4485.506548] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4490.678069] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4497.346528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4497.805110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4498.804317] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4502.897176] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4503.999635] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4511.825562] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4523.490400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4523.499917] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4523.510479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4523.531063] Lustre: Skipped 1 previous similar message [ 4526.665783] Lustre: server umount lustre-OST0000 complete [ 4533.733459] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4533.737905] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4533.748040] Lustre: Skipped 1 previous similar message [ 4533.755876] Lustre: Skipped 2 previous similar messages [ 4537.027128] Lustre: server umount lustre-MDT0000 complete [ 4538.857965] LustreError: 157888:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4538.891447] LustreError: 157888:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4540.792680] LustreError: 157871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782224808 with bad export cookie 1883560085783540442 [ 4540.807380] LustreError: 157871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4540.817904] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4541.185790] Lustre: server umount lustre-MDT0001 complete [ 4551.141583] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4555.780738] Key type lgssc unregistered [ 4556.383232] LNet: 161436:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4556.396901] LNetError: 161436:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4556.457850] LNet: Removed LNI 192.168.204.106@tcp [ 4557.936063] Key type .llcrypt unregistered [ 4557.939200] Key type ._llcrypt unregistered [ 4587.167474] Key type ._llcrypt registered [ 4587.169889] Key type .llcrypt registered [ 4587.283805] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4588.116002] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4588.238471] alg: No test for adler32 (adler32-zlib) [ 4589.402157] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 4589.722052] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4591.384316] Key type lgssc registered [ 4592.848797] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4611.297558] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:27:57 (1782224877) [ 4628.570553] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 4640.597799] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4677.791074] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4677.839527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4679.069163] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4679.095708] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4679.159341] Lustre: lustre-MDT0000: new disk, initializing [ 4679.263717] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4679.282501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4683.791724] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4695.120119] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4695.250358] Lustre: 165713:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4695.302572] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4695.306281] Lustre: Skipped 1 previous similar message [ 4695.390931] Lustre: lustre-MDT0001: new disk, initializing [ 4695.543714] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4695.599115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4695.613489] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4700.632593] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4710.405888] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4716.618271] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4724.048270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4724.450357] Lustre: lustre-OST0000: new disk, initializing [ 4724.453883] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4724.459391] Lustre: 167684:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4724.560790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4731.186652] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4732.460164] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4732.472120] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4732.614614] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4740.219178] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4753.395337] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4753.400666] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4753.406442] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4753.443147] Lustre: Skipped 1 previous similar message [ 4755.278729] Lustre: server umount lustre-OST0000 complete [ 4762.082237] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4762.105239] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4762.120515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4762.130358] Lustre: Skipped 1 previous similar message [ 4763.619696] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4763.622317] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4765.756528] Lustre: server umount lustre-MDT0000 complete [ 4769.921075] LustreError: 166518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782225037 with bad export cookie 6627615652673928336 [ 4769.943241] LustreError: 166518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4769.944644] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4770.383654] Lustre: server umount lustre-MDT0001 complete [ 4782.323656] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4785.684237] Key type lgssc unregistered [ 4786.024468] LNet: 169589:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4786.039922] LNetError: 169589:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4787.129833] LNet: Removed LNI 192.168.204.106@tcp [ 4788.291582] Key type .llcrypt unregistered [ 4788.293281] Key type ._llcrypt unregistered [ 4813.619259] Key type ._llcrypt registered [ 4813.622418] Key type .llcrypt registered [ 4813.768895] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4814.914917] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4814.979508] alg: No test for adler32 (adler32-zlib) [ 4816.054794] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 4816.328499] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4817.984160] Key type lgssc registered [ 4819.269344] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4827.432439] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4827.476980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4828.989332] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4833.672739] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4838.375446] LustreError: 171005:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4838.401882] LustreError: 171005:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4842.297620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4842.577337] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4847.984732] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4856.054257] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4861.642622] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4868.554631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4868.928503] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4874.042798] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4874.574976] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4880.520403] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4892.344212] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4892.392602] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4892.407129] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4892.432603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.461683] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4893.736340] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4893.764834] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4893.773260] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4893.784893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.794641] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4894.828738] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4894.852775] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4894.864104] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4894.870780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.883519] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4896.130301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.496985] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4897.503745] Lustre: Skipped 1 previous similar message [ 4897.534921] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4897.550144] Lustre: Skipped 1 previous similar message [ 4897.560088] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4897.574679] Lustre: Skipped 1 previous similar message [ 4897.582517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.608528] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4897.620042] Lustre: Skipped 1 previous similar message [ 4898.834395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.023732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.999992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.117811] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4902.128982] Lustre: Skipped 3 previous similar messages [ 4902.150729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4902.170400] Lustre: Skipped 3 previous similar messages [ 4902.174377] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4902.184242] Lustre: Skipped 3 previous similar messages [ 4902.191398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.205660] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4902.207886] Lustre: Skipped 3 previous similar messages [ 4903.348386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.423087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.151903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.940416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.766551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.568403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.458848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.217139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.044838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.156474] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4911.163142] Lustre: Skipped 9 previous similar messages [ 4911.195240] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4911.218592] Lustre: Skipped 9 previous similar messages [ 4911.247913] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4911.265613] Lustre: Skipped 9 previous similar messages [ 4911.281163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.311935] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4911.334724] Lustre: Skipped 9 previous similar messages [ 4912.194244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.472429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.487248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.360240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.342747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.597094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.893179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.427680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.667367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.908391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.788628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.598922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.674766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.604898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.556476] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4927.581460] Lustre: Skipped 14 previous similar messages [ 4927.608525] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4927.628766] Lustre: Skipped 14 previous similar messages [ 4927.635931] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4927.651540] Lustre: Skipped 14 previous similar messages [ 4927.661368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.673017] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4927.688203] Lustre: Skipped 14 previous similar messages [ 4928.659087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.712389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.763454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.764410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.930874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.129279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.399754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.281107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.234500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.317941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.229612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.196739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.161470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.207845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.216044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.131636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.977613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.912717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.748744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.663467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.529724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.516648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.308639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.153679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.980618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.861748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.732532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.643054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.688918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.672806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.662967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.465166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.498536] LustreError: 171024:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4959.511520] LustreError: 171024:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4959.530885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.401459] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4960.408199] Lustre: Skipped 33 previous similar messages [ 4960.422159] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4960.433963] Lustre: Skipped 33 previous similar messages [ 4960.438395] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4960.443329] Lustre: Skipped 33 previous similar messages [ 4960.451257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.462073] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4960.471400] Lustre: Skipped 33 previous similar messages [ 4961.472719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.463522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.493378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.346733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.254071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.127068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.917289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.649626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.471689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.254550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.217236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.006819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.793489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.545431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.413308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.266767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.142405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.042615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.870577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.849983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.871291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.708911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.558837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.461275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.405584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.347685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.250039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.044775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.916523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.788333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.810594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.691507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.508615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.297595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.052589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.988603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.850876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.553064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.403850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.195178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.008989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.812855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.744253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.574542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.550225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.396571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.299762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.106463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.890910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.609192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.573751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.412249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.369285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.193144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.067974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.911988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.913497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.911735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.923180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.854127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.953735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.109352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.116889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.211911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.012613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.059358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.049877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.969176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.722289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.658264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.578911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.638867] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5024.647412] Lustre: Skipped 71 previous similar messages [ 5024.659622] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5024.677914] Lustre: Skipped 71 previous similar messages [ 5024.681241] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5024.688729] Lustre: Skipped 71 previous similar messages [ 5024.695129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.724516] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5024.728630] Lustre: Skipped 71 previous similar messages [ 5025.640522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.774309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.864289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.755954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.637960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.417602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.081384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.814526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.609561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.261324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.025872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.782464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.671181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.499514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.456356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.244192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.894437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.660444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.308642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.905543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.656168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.405846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.064359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.599303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.460308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.207540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.886805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.758978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.611845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.569792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.429460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.306303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.058595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.967851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.878818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.036522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.912622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.881819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.726994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.755453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.730362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.769530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.707033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.561606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.445547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.381178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.236366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.226122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.099847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.195636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.126340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.134840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.116925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.126697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.017662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.031628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.042077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.893683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.893584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.878189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.733557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.687881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.580302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.374085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.160132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.093463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.932134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.732858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.556699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.319125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.298505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.115627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.885772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.852267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.797395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.593853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.520518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.834812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.967336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.264722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.287718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.300522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.102782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.147762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.253734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.410497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.272241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.254919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.464208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.666649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.623935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.659994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.764754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.562968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.713239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.572555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.485090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.272252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.491076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.343236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.228874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.126404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.012947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.927328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.996188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.000394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.904553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.871886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.866351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.795941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.826643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.645614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.464035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.374181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.310437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.203585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.118399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.860607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.759702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.592790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.578027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.441179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.382276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.252292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.133181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.952484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.910274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.880716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.948905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.846714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.850184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.878640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.792739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.699554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.565250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.395168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.407652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.348872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.212367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.113738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.014169] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5153.019727] Lustre: Skipped 140 previous similar messages [ 5153.030444] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5153.039747] Lustre: Skipped 140 previous similar messages [ 5153.043934] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5153.048748] Lustre: Skipped 140 previous similar messages [ 5153.052321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.066569] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5153.072606] Lustre: Skipped 140 previous similar messages [ 5153.992045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.942626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.847696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.770416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.625624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.562682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.498548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.432276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.401712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.367248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.362873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.163498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.114534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.009217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.870817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.581260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.384362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.396606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.354078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.307375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.284133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.228805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.088814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.899231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.764770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.709144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.649180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.486235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.329224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.194748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.088250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.039050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.023644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.966926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.064554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.981927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.920844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.021357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.799626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.704827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.678181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.723523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.518969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.391617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.441181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.301528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.244422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.159975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.090336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.918382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.801151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.721321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.792662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.739331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.752547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.721946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.692347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.658927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.523584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.333432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.313244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.087817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.948079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.839432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.694390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.437438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.297222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.030903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.890265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.710773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.584583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.398129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.290149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.069705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.857028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.626691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.490421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.471822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.477459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.430674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.423500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.273614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.298307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.386914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.355078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.272054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.251195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.101729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.959390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.875577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.743995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.567649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.442041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.255895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.160642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.153111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.106839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.987719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.873694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.850815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.718797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.770244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.720238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.616618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.517924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.218035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.050240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.939501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.866377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.084761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.947130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.812645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.708122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.695498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.630462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.576089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.585078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.476681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.254205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.113515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.917135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.766642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.649121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.636420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.503106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.521817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.446669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.604837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.665985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.587290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.507891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.586510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.605631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.604235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.582403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.705562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.453497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.353735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.174442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.079309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.905862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.628294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.324117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.133056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.992040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.873235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.828942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.744674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.680621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.831220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.605816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.789800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.074734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.056972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.858675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.461386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.554978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.757603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.505637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.448568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.407227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.340701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.224583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.203257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.149579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.990362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.869884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.799745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.563672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.453154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.169722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.155563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.987366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.871844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.689611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.752537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.647069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.518516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.444652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.309241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.462344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.390252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.336653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.238795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.150372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.991872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.908278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.703042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.570255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.466868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.401843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.249815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.316827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.132545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.051370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.924937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.828781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.748114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.667645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.616459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.706709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.605307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.555871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.501358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.502950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.499863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.400436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.335947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.285729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.083904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.062460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.877523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.709478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.636572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.486910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.394404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.165877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.005147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.873979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.770990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.708117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.700684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.617405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.426208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.202884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.163341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.990213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.911648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.644164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.402480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.267205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.080439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.916968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.678164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.375380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.200344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.114739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.019929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.958342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.067811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.030502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.114704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.048690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.211662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.165735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.077479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.937097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.717934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.540437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.462187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.308625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.219988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.931680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.864352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.781852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.742036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.626544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.722051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.736352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.822294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.852894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.795459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.777600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.662036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.660496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.574549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.608918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.590136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.755565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.780409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.953206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.931660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.909852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.867866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.815176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.772563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.832714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.570412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.499330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.389315] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5409.392432] Lustre: Skipped 279 previous similar messages [ 5409.412294] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5409.425211] Lustre: Skipped 279 previous similar messages [ 5409.430494] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5409.436711] Lustre: Skipped 279 previous similar messages [ 5409.446241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.459369] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5409.464974] Lustre: Skipped 279 previous similar messages [ 5410.477360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.377707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.315997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.168114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.991094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.711447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.513276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.249533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.936370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.653435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.460525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.329131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.276620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.191561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.909059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.849271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.801908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.622066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.719316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.063265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.388030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.711993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.014519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.018905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.726446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.650076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.695926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.848657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.984708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.019567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.032897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.231921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.582919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.627041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.561671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.757776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.067751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.971459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.820677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.131193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.075528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.065820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.387299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.631155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.661629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.104950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.170354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.369427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.872030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.557805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.395033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.160724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.535928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.578156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.392732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.327653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.476428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.690714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.373593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.623552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.498043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.679078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.421123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.221301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.789931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.534033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.315932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.062805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.836634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.489497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.107170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.809919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.535919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.476823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.328761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.368130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.327855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.205107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.280203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.279871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.354673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.233249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.191237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.923352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.887900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.853803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.879193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.873058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.009286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.042519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.043065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.011540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.013788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.808154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.683608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.559886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.427470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.409263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.242409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.313633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.406563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.540662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.549344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.625430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.672592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.673987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.765266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.716175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.805729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.711362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.782472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.637510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.702483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.706515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.820792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.958288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.835550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.783319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.779798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.701480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.765072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.570413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.610364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.622391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.424256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.301944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.212931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.076863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.961287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.952387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.017210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.797862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.757811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.584488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.346592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.167879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.983093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.872663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.770780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.610861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.480964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.394631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.230745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.026742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.918152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.859629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.821055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.703619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.512854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.529348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.316198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.201158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.901336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.786632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.677692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.641155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.468399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.261427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.038445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.005771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.930203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.898652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.756215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.709589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.672322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.603812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.409090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.313043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.215378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.064498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.919709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.688427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.604472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.576212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.455571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.385681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.465233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.330069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.394716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.349423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.337485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.376205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.400057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.345899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.376279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.258557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.168599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.121881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.046977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.080993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.071179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.136853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.110883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.040684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.902203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.698934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.587449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.477812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.337213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.267398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.115165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.852044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.611604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.290789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.147441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.027496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.906816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.797721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.597236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.430154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.432239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.329111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.201132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.097731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.024260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.970082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.811131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.648764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.544942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.477097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.350521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.252946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.226504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.211973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.062312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.954276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.908403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.783746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.644681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.415928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.342870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.301592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.511871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.592614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.609489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.597309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.708894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.827476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.642174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.606060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.511475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.521467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.659071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.507745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.395783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.178785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.130497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.062636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.034525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.067734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.882398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.834064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.718893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.626162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.783343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.718571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.768412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.733101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.703842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.657544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.778074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.839800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.666178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.645430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.634092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.662043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.568307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.581447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.482278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.517586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.359859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.257437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.295906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.215024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.134397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.077717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.904742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.821336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.708801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.853356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.704887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.687697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.651586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.615896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.490280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.509611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.414161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.360914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.173471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.100881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.025543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.992964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.884077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.776849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.667296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.442781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.367528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.317614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.090596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.935237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.719343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.432307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.268369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.105464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.076546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.910795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.724771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.526578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.516549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.468719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.312491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.378869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.223104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.017131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.774254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.552949] LustreError: 171024:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5705.560346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.561837] LustreError: 171024:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5706.379435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.284790] LustreError: 171024:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5707.298836] LustreError: 171024:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5707.308805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.085653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.978055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.764790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.608943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.407672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.243799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.188492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.094780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.033883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.136664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.192628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.133675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.110550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.191637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.098781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.087988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.885316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.010957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.829159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.721984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.532715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.321951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.117519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.845964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.625808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.526747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.348250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.113628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.968844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.830551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.718217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.574696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.562813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.511985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.519554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.354260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.284041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.159160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.032931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.803699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.720646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.528660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.430632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.266378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.136392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.009878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.934168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.703689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.507994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.397421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.196951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.137176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.995350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.882754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.880455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.880587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.846676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.784927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.604334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.452509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.281333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.241574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.110537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.013360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.879898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.834940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.821470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.705942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.685302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.672206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.538570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.491477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.527416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.290457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.242478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.024753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.974163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.932133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.803685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.491556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.283890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.935208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.829796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.638378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.671084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.566548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.421358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.172638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.836640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.682316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.494641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.414234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.475473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.357626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.402877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.551355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.649555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.101437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.018311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.971481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.947458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.959707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.100101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.409663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.951756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.005836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.480784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.424538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.664820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.757828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.787068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.687786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.565692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.561315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.421960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.287086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.978614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.822934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.718274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.582859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.628295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.204229] LustreError: 173148:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6178.210287] LustreError: 171024:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6832.102023] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6832.104365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6832.121644] Lustre: Skipped 440 previous similar messages [ 6837.219062] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6837.227790] Lustre: Skipped 2 previous similar messages [ 6837.742535] Lustre: server umount lustre-OST0000 complete [ 6842.346961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6842.363601] Lustre: Skipped 1 previous similar message [ 6847.463353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6852.580713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6852.588645] Lustre: Skipped 2 previous similar messages [ 6856.160136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6856.533243] Lustre: server umount lustre-MDT0000 complete [ 6857.704288] LustreError: 171000:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6857.727961] LustreError: 171000:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6860.296918] LustreError: 170985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782227127 with bad export cookie 9084408937347554180 [ 6860.309589] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6860.642437] Lustre: server umount lustre-MDT0001 complete [ 6871.168162] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 6874.029507] Key type lgssc unregistered [ 6874.366521] LNet: 226264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6874.388596] LNetError: 226264:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6874.409467] LNet: Removed LNI 192.168.204.106@tcp [ 6875.381107] Key type .llcrypt unregistered [ 6875.383318] Key type ._llcrypt unregistered [ 6901.512358] Key type ._llcrypt registered [ 6901.521664] Key type .llcrypt registered [ 6901.679840] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 6902.960970] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6903.066345] alg: No test for adler32 (adler32-zlib) [ 6904.147824] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 6904.483653] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 6906.176173] Key type lgssc registered [ 6907.275990] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6916.039973] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6916.066641] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6917.596974] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6922.609769] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6927.847547] LustreError: 227689:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6927.873747] LustreError: 227689:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6931.410360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6931.841432] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6936.590466] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6944.548756] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6950.758610] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6958.258375] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6958.680401] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6962.822088] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6965.276726] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6972.911773] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6981.676751] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6983.635918] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6983.657303] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6983.671438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.692710] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6983.693378] LustreError: 227708:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6985.699850] Lustre: lustre-OST0000: Not available for connect from 192.168.204.6@tcp (stopping) [ 6987.745970] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6987.750165] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6987.785610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6988.776702] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6988.815479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6988.819736] Lustre: Skipped 1 previous similar message [ 6991.874524] Lustre: server umount lustre-OST0000 complete [ 7003.768129] LustreError: 227708:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7026.583694] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7026.948591] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7029.046822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.068246] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7033.282550] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7038.309860] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7038.339551] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7040.788512] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7042.511551] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7047.941159] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b908770a800.ost_server_uuid 50 [ 7049.489443] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b908770a800.ost_server_uuid in FULL state after 0 sec [ 7054.839837] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7054.847467] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7054.860735] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7054.870884] Lustre: Skipped 1 previous similar message [ 7056.337377] Lustre: server umount lustre-OST0000 complete [ 7064.046235] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7064.055039] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7064.059249] Lustre: Skipped 1 previous similar message [ 7064.077876] Lustre: Skipped 1 previous similar message [ 7066.566815] Lustre: server umount lustre-MDT0000 complete [ 7069.155710] LustreError: 227689:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7069.190772] LustreError: 227689:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7070.200604] LustreError: 227668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782227337 with bad export cookie 7578281190296346463 [ 7070.204541] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7070.221016] LustreError: 227668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7079.583996] Lustre: server umount lustre-MDT0001 complete [ 7091.023475] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 7094.120776] Key type lgssc unregistered [ 7094.478674] LNet: 232898:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7094.489567] LNetError: 232898:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7094.514924] LNet: Removed LNI 192.168.204.106@tcp [ 7095.400488] Key type .llcrypt unregistered [ 7095.406292] Key type ._llcrypt unregistered [ 7126.480614] Key type ._llcrypt registered [ 7126.483990] Key type .llcrypt registered [ 7126.655630] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7127.679387] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7127.696915] alg: No test for adler32 (adler32-zlib) [ 7128.699394] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 7128.997333] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 7130.752200] Key type lgssc registered [ 7131.891146] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7140.115745] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7140.138132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7141.763585] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7146.135597] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7152.101661] LustreError: 234640:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7152.133352] LustreError: 234640:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7154.254881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7154.672190] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7158.526571] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7161.445459] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7168.813060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7169.284082] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7176.272987] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7185.506274] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7185.676421] Lustre: lustre-OST0001: new disk, initializing [ 7185.681709] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7185.691443] Lustre: 236949:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7185.786760] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7186.896624] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7192.178357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7192.200044] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7192.340372] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7193.116545] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7203.933767] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7210.605456] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7226.338473] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7226.351519] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7226.377480] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7228.387120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7228.402153] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7228.458954] Lustre: Skipped 2 previous similar messages [ 7231.299237] Lustre: server umount lustre-MDT0000 complete [ 7233.509376] LustreError: 234641:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7233.526617] LustreError: 234641:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7234.673719] LustreError: 237717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782227502 with bad export cookie 7420697798478241707 [ 7234.676562] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7234.685343] LustreError: 237717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7234.991866] Lustre: server umount lustre-MDT0001 complete [ 7249.245513] Lustre: server umount lustre-OST0000 complete [ 7262.809506] Lustre: server umount lustre-OST0001 complete [ 7282.962159] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7292.984524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7293.335899] LustreError: 240053:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7293.409070] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7297.598913] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7298.540704] LustreError: 240054:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7303.655839] LustreError: 240053:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7305.505545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7305.876136] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7310.356973] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7313.224567] Lustre: 241195:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7319.816073] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7326.454561] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7330.346577] LustreError: 241548:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7334.686929] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7334.885745] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7334.894077] Lustre: Skipped 1 previous similar message [ 7339.024078] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7339.027849] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7340.949276] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7348.194847] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7351.801697] Lustre: 243064:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7362.018695] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7362.026046] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7362.040919] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7362.044450] Lustre: Skipped 3 previous similar messages [ 7364.587057] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7364.591310] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7364.601584] Lustre: Skipped 2 previous similar messages [ 7364.619956] Lustre: Skipped 3 previous similar messages [ 7369.708114] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7369.713612] Lustre: Skipped 3 previous similar messages [ 7376.352575] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7376.480597] Lustre: server umount mylustre-MDT0000 complete [ 7379.941615] LustreError: 240050:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7379.978435] LustreError: 240050:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7380.015824] LustreError: 240030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782227647 with bad export cookie 7420697798478244514 [ 7380.031986] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7380.369907] Lustre: server umount mylustre-MDT0001 complete [ 7394.575927] Lustre: server umount mylustre-OST0000 complete [ 7409.346708] Lustre: server umount mylustre-OST0001 complete [ 7430.623418] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7441.232959] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7441.842103] LustreError: 245375:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7441.926774] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7445.975951] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7454.025739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7458.904665] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7461.418344] Lustre: 246519:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7467.637714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7473.648213] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7481.711304] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7487.936716] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7488.009690] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7488.009946] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7495.216143] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7499.067421] Lustre: 248389:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7508.450123] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7508.465564] Lustre: Skipped 1 previous similar message [ 7508.467566] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7508.487225] Lustre: Skipped 7 previous similar messages [ 7511.910718] Lustre: server umount tfs-MDT0000 complete [ 7513.571398] LustreError: 245372:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7513.587790] LustreError: 245372:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 7515.485794] LustreError: 246899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782227783 with bad export cookie 7420697798478246404 [ 7515.486712] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7515.497515] LustreError: 246899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7515.845769] Lustre: server umount tfs-MDT0001 complete [ 7529.755692] Lustre: server umount tfs-OST0000 complete [ 7544.189985] Lustre: server umount tfs-OST0001 complete [ 7566.179522] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7579.018830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7579.553734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7579.569463] Lustre: Skipped 3 previous similar messages [ 7585.002857] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7596.868954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7602.125842] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7605.361129] Lustre: 251784:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7612.513934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7619.950477] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7625.210949] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7629.002721] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7634.438644] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7635.353126] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7642.779357] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7646.367665] Lustre: 253656:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7652.504049] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:18:38 (1782227918) [ 7669.218785] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7669.245722] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7669.279142] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7675.051722] Lustre: server umount lustre-MDT0000 complete [ 7675.383253] LustreError: 250636:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7675.402485] LustreError: 250636:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 10 previous similar messages [ 7678.705130] LustreError: 250615:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782227946 with bad export cookie 7420697798478248070 [ 7678.718221] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7693.281831] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7693.603234] Lustre: server umount lustre-MDT0001 complete [ 7711.200285] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7711.326650] Lustre: server umount lustre-OST0000 complete [ 7721.287444] Lustre: server umount lustre-OST0001 complete [ 7727.893051] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 7737.394142] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7775.354260] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7775.354260] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7775.354260] [ 7775.432946] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7775.593270] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7775.612697] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7775.682717] Lustre: lustre-MDT0000: new disk, initializing [ 7775.762228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7775.767172] Lustre: Skipped 3 previous similar messages [ 7775.782863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7780.064634] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7791.405370] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7791.405370] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7791.405370] [ 7791.474989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7791.626900] Lustre: 256844:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7791.691734] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7791.695875] Lustre: Skipped 1 previous similar message [ 7791.784435] Lustre: lustre-MDT0001: new disk, initializing [ 7791.950996] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7791.975536] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7796.894478] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7808.204357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7808.542612] Lustre: lustre-OST0000: new disk, initializing [ 7808.546575] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7808.554130] Lustre: 258425:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7810.043516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7810.051095] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7810.171875] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7815.368591] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7826.922918] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7827.132264] Lustre: lustre-OST0001: new disk, initializing [ 7827.138045] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7827.143301] Lustre: 259298:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7828.542449] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7828.553051] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7828.663414] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7833.730626] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7845.287936] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:21:51 (1782228111) [ 7859.746254] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:22:05 (1782228125) [ 7875.048514] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7875.061390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7875.077231] Lustre: Skipped 6 previous similar messages [ 7875.079040] Lustre: Skipped 10 previous similar messages [ 7880.852238] Lustre: server umount lustre-OST0000 complete [ 7891.176935] Lustre: server umount lustre-MDT0000 complete [ 7895.241603] LustreError: 256836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782228162 with bad export cookie 7420697798478249624 [ 7895.256923] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7895.274979] LustreError: 256836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7901.787296] Lustre: server umount lustre-MDT0001 complete [ 7913.482759] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 7924.571804] Lustre: server umount lustre-OST0001 complete [ 7930.897542] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 7945.588142] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7983.127735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7983.325378] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7983.356540] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7983.428093] Lustre: lustre-MDT0000: new disk, initializing [ 7983.520406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7988.237461] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7999.964644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8000.192511] Lustre: 263855:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8000.308226] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8000.313183] Lustre: Skipped 1 previous similar message [ 8000.417964] Lustre: lustre-MDT0001: new disk, initializing [ 8000.561981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8000.573155] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8005.220560] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8014.490330] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8020.761786] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8028.254842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8028.684687] Lustre: 265823:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8030.461694] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8030.618330] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8036.177844] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8046.603248] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8056.294450] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8056.305021] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8056.308473] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8056.308481] Lustre: Skipped 11 previous similar messages [ 8056.315517] Lustre: Skipped 5 previous similar messages [ 8056.355448] LustreError: Skipped 1 previous similar message [ 8059.215045] Lustre: server umount lustre-OST0000 complete [ 8069.799590] Lustre: server umount lustre-MDT0000 complete [ 8071.654605] LustreError: 263862:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8071.680466] LustreError: 263862:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 8074.158613] LustreError: 264661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782228341 with bad export cookie 7420697798478251094 [ 8074.171397] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8074.175030] LustreError: 264661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8074.611127] Lustre: server umount lustre-MDT0001 complete [ 8086.821148] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8090.009521] Key type lgssc unregistered [ 8090.334419] LNet: 267584:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8090.339237] LNetError: 267584:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8090.353627] LNet: Removed LNI 192.168.204.106@tcp [ 8091.298574] Key type .llcrypt unregistered [ 8091.307221] Key type ._llcrypt unregistered [ 8125.418452] Key type ._llcrypt registered [ 8125.422827] Key type .llcrypt registered [ 8125.573267] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8127.094149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8127.154341] alg: No test for adler32 (adler32-zlib) [ 8128.381625] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 8128.649297] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8130.353345] Key type lgssc registered [ 8131.798829] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8141.477581] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8141.507062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8143.254340] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8148.853182] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8152.573044] LustreError: 269048:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8152.590904] LustreError: 269048:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8157.667843] LustreError: 269049:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8159.770166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8160.427356] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8166.209907] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8176.678565] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8185.500438] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8195.459596] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8196.274112] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8203.673426] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8211.801423] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8212.666729] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8213.206916] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8218.087884] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8218.102581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8218.108204] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8221.825934] Lustre: server umount lustre-OST0000 complete [ 8226.785952] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8226.794853] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8226.807544] Lustre: Skipped 1 previous similar message [ 8226.819870] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8226.822675] Lustre: Skipped 1 previous similar message [ 8228.323101] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8228.328480] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8232.582453] Lustre: server umount lustre-MDT0000 complete [ 8233.440960] LustreError: 269049:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8233.462968] LustreError: 269049:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8236.368337] LustreError: 269030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782228504 with bad export cookie 14202827270826592011 [ 8236.380307] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8236.381209] LustreError: 269030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8236.774773] Lustre: server umount lustre-MDT0001 complete [ 8247.342477] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8250.653449] Key type lgssc unregistered [ 8250.984577] LNet: 272635:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8250.992386] LNetError: 272635:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8252.060044] LNet: Removed LNI 192.168.204.106@tcp [ 8252.881111] Key type .llcrypt unregistered [ 8252.884477] Key type ._llcrypt unregistered [ 8272.066699] Key type ._llcrypt registered [ 8272.069459] Key type .llcrypt registered [ 8272.223707] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8273.442310] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8273.651659] alg: No test for adler32 (adler32-zlib) [ 8274.924267] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 8275.241730] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8276.932118] Key type lgssc registered [ 8278.040716] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8287.803587] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8287.832337] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8303.456786] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8308.576525] LustreError: 273833:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.106@tcp: failed processing log, type 4: rc = -110 [ 8335.330940] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8341.752879] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8348.504333] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8350.496474] Lustre: server umount lustre-OST0000 complete [ 8364.720353] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8367.820512] Key type lgssc unregistered [ 8368.160273] LNet: 275464:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8368.184083] LNetError: 275464:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8369.256171] LNet: Removed LNI 192.168.204.106@tcp [ 8370.029359] Key type .llcrypt unregistered [ 8370.036686] Key type ._llcrypt unregistered [ 8407.614987] Key type ._llcrypt registered [ 8407.623961] Key type .llcrypt registered [ 8407.735575] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8408.782209] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8408.929450] alg: No test for adler32 (adler32-zlib) [ 8410.073733] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 8410.309737] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8411.968257] Key type lgssc registered [ 8413.590561] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8427.016237] loop0: detected capacity change from 0 to 204800000 [ 8427.222714] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8427.246842] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8427.767169] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8428.782168] loop0: detected capacity change from 0 to 204800000 [ 8429.032606] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8429.045282] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8429.319061] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8430.248564] loop0: detected capacity change from 0 to 204800000 [ 8430.488120] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8430.496463] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8430.587562] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8432.157717] loop0: detected capacity change from 0 to 204800000 [ 8432.286663] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8432.304889] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8432.428399] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8433.512677] loop0: detected capacity change from 0 to 204800000 [ 8433.550731] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8438.632650] loop0: detected capacity change from 0 to 204800000 [ 8438.684873] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8443.326507] loop0: detected capacity change from 0 to 204800000 [ 8443.358107] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8447.157582] loop0: detected capacity change from 0 to 204800000 [ 8447.201754] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8451.557542] loop0: detected capacity change from 0 to 204800000 [ 8451.652568] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8451.745842] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8451.841742] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8451.851687] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8451.864410] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8451.882394] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8451.892149] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8451.900967] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8451.911599] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8451.919118] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8451.925995] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8451.931811] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8451.945647] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8451.956688] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8451.977043] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8451.990407] Lustre: 278879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8457.591745] Lustre: server umount MGS complete [ 8458.589735] loop0: detected capacity change from 0 to 204800000 [ 8458.688688] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8459.308088] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8459.353565] Lustre: lustre-MDT0000: Aborting client recovery [ 8460.172523] loop1: detected capacity change from 0 to 204800000 [ 8460.279139] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8460.414265] Lustre: 279310:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8460.436584] Lustre: 279310:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8460.480268] Lustre: 279310:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8460.506888] Lustre: 279310:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8460.518308] Lustre: 279310:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8460.532698] Lustre: 279310:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8460.541756] Lustre: 279310:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8460.560132] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8460.763283] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8460.815731] Lustre: lustre-MDT0001: Aborting client recovery [ 8461.792636] loop2: detected capacity change from 0 to 204800000 [ 8461.953313] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8462.006872] Lustre: 279389:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8462.013881] Lustre: 279389:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8462.022499] Lustre: 279389:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8462.030882] Lustre: 279389:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8462.037822] Lustre: 279389:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8462.041987] Lustre: 279389:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8462.049557] Lustre: 279389:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8462.169838] Lustre: Found index 0 for lustre-OST0000, updating log [ 8462.202773] Lustre: 279407:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8462.291250] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8462.328824] Lustre: lustre-OST0000: Aborting recovery [ 8463.289446] loop3: detected capacity change from 0 to 204800000 [ 8463.464973] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8463.633897] Lustre: 279473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8463.657080] Lustre: 279473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8463.675788] Lustre: 279473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8463.693179] Lustre: 279473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8463.705755] Lustre: 279473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8463.717584] Lustre: 279473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8463.730660] Lustre: 279473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8463.760603] Lustre: Found index 1 for lustre-OST0001, updating log [ 8463.817520] Lustre: 279478:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8463.925566] Lustre: lustre-OST0001: Aborting recovery [ 8469.027472] LustreError: 279254:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8470.006668] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8470.062155] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8470.073671] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8470.157453] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8471.017770] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8471.022601] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8473.768522] LustreError: 279221:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1370us: rc = -2 [ 8474.269158] LustreError: 279222:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 915us: rc = -2 [ 8474.301586] LustreError: 279222:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 8491.488526] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8491.489362] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8491.519792] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8491.525088] Lustre: Skipped 2 previous similar messages [ 8496.611280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8496.624284] Lustre: Skipped 3 previous similar messages [ 8501.216195] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8501.579524] Lustre: server umount lustre-MDT0000 complete [ 8501.732916] LustreError: 279708:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8501.759627] LustreError: 279708:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 8502.739675] LustreError: 279206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782228770 with bad export cookie 13754264048620112840 [ 8502.753525] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8502.757172] LustreError: 279206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8506.850416] LustreError: 279218:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8506.854850] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8506.856790] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8506.881268] LustreError: 279218:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8506.901701] Lustre: Skipped 4 previous similar messages [ 8507.873116] LustreError: 279219:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8507.907079] LustreError: 279219:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8513.001142] LustreError: 279218:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8513.001147] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8513.020197] Lustre: Skipped 4 previous similar messages [ 8517.089818] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8517.779994] Lustre: server umount lustre-MDT0001 complete [ 8525.128327] Lustre: server umount lustre-OST0000 complete [ 8532.284331] Lustre: server umount lustre-OST0001 complete [ 8539.713835] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:33:26 (1782228806) [ 8565.595223] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 8574.563550] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8604.774137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8604.996976] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8605.026246] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8605.104130] Lustre: lustre-MDT0000: new disk, initializing [ 8605.198712] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8605.203113] Lustre: Skipped 1 previous similar message [ 8605.216384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8609.187809] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8620.043206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8620.190989] Lustre: 283543:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8620.272765] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8620.284187] Lustre: Skipped 1 previous similar message [ 8620.375394] Lustre: lustre-MDT0001: new disk, initializing [ 8620.588135] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8620.662814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8620.681495] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8625.642661] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8635.091773] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8641.200503] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8649.065916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8649.356927] Lustre: lustre-OST0000: new disk, initializing [ 8649.368314] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8649.390842] Lustre: 285512:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8649.487972] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8655.786655] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8656.516394] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8656.525876] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8656.700722] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8665.023856] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8669.450520] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8678.288560] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8683.613204] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8728.561077] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8728.563442] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8728.567556] LustreError: Skipped 1 previous similar message [ 8728.585651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8728.605197] Lustre: Skipped 1 previous similar message [ 8738.793159] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8738.796939] Lustre: Skipped 3 previous similar messages [ 8739.296129] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8739.399993] Lustre: server umount lustre-OST0000 complete [ 8743.398267] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8743.413418] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8743.431902] Lustre: Skipped 1 previous similar message [ 8757.216135] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8757.427831] Lustre: server umount lustre-MDT0000 complete [ 8759.265745] LustreError: 283554:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8759.298619] LustreError: 283554:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8761.097141] LustreError: 283534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782229028 with bad export cookie 13754264048620115549 [ 8761.106334] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8761.110396] LustreError: 283534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8761.503822] Lustre: server umount lustre-MDT0001 complete [ 8768.939286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8772.974914] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8777.755509] Lustre: server umount MGS complete [ 8788.517912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8789.015879] LustreError: 289326:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8789.047573] LustreError: 289339:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8789.102293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8794.107692] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8802.010702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8806.473286] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8814.127945] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8819.795357] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8825.554034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8825.927691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8825.943628] Lustre: Skipped 1 previous similar message [ 8832.372323] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8842.006907] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8873.466956] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8873.487343] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8873.500786] Lustre: Skipped 1 previous similar message [ 8873.504772] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8873.507941] Lustre: Skipped 6 previous similar messages [ 8877.457952] Lustre: server umount lustre-OST0000 complete [ 8884.196900] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8884.205367] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8884.224918] Lustre: Skipped 1 previous similar message [ 8888.293188] Lustre: server umount lustre-MDT0000 complete [ 8890.849316] LustreError: 289327:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8890.895377] LustreError: 289327:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8893.125817] LustreError: 292166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782229160 with bad export cookie 13754264048620117397 [ 8893.133630] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8893.148705] LustreError: 292166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8893.669308] Lustre: server umount lustre-MDT0001 complete [ 8902.878683] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8905.384095] Key type lgssc unregistered [ 8905.652799] LNet: 293273:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8905.658549] LNetError: 293273:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8905.669558] LNet: Removed LNI 192.168.204.106@tcp [ 8906.365094] Key type .llcrypt unregistered [ 8906.367254] Key type ._llcrypt unregistered [ 8941.434836] Key type ._llcrypt registered [ 8941.436787] Key type .llcrypt registered [ 8941.524670] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 8957.197909] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8958.446414] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8958.465709] alg: No test for adler32 (adler32-zlib) [ 8959.495431] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 8959.728597] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8961.434796] Key type lgssc registered [ 8962.586922] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8996.106968] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8996.156960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8997.400914] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8997.423418] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8997.517643] Lustre: lustre-MDT0000: new disk, initializing [ 8997.590610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8997.607735] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9001.675516] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9011.621955] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9011.751564] Lustre: 297235:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9011.817428] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9011.822920] Lustre: Skipped 1 previous similar message [ 9011.903813] Lustre: lustre-MDT0001: new disk, initializing [ 9012.010085] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9012.048920] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9012.058730] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9015.818452] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9024.544344] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9029.777755] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9037.229348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9037.593214] Lustre: lustre-OST0000: new disk, initializing [ 9037.603139] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9037.612029] Lustre: 299203:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9037.719701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9043.932347] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9045.588187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9045.594339] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9045.735252] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9053.211215] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9061.222142] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9070.536654] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9078.196761] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9087.127960] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9133.032385] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9133.041398] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9133.059606] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9133.077525] Lustre: Skipped 1 previous similar message [ 9136.555703] Lustre: server umount lustre-OST0000 complete [ 9143.270566] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9143.271562] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9143.282778] Lustre: Skipped 2 previous similar messages [ 9146.866367] Lustre: server umount lustre-MDT0000 complete [ 9150.442839] LustreError: 298029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782229418 with bad export cookie 12261470476885069060 [ 9150.444911] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9150.451125] LustreError: 298029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9150.754718] Lustre: server umount lustre-MDT0001 complete [ 9158.870652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9163.629914] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9169.523838] Lustre: server umount MGS complete [ 9181.517964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9182.001041] LustreError: 303074:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9182.022153] LustreError: 303074:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9182.032106] LustreError: 303087:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9182.116479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9187.302235] LustreError: 303075:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9187.339185] LustreError: 303075:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9187.407971] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9191.396683] LustreError: 303074:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9196.436328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9196.523506] LustreError: 303075:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9196.894525] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9201.404671] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9209.129794] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9214.353372] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9220.899305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9221.263226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9227.271085] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9234.916535] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9262.532341] Lustre: server umount lustre-OST0000 complete [ 9267.687301] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9267.693771] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9267.700243] Lustre: Skipped 1 previous similar message [ 9267.718074] Lustre: Skipped 2 previous similar messages [ 9272.803967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9272.810754] Lustre: Skipped 1 previous similar message [ 9276.978020] Lustre: server umount lustre-MDT0000 complete [ 9277.927346] LustreError: 303074:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9277.965694] LustreError: 303074:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9278.944137] Lustre: 295458:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1782229530/real 1782229530] req@000000000d6a89c6 x1868802794079744/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1782229546 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 9278.978456] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9279.968124] Lustre: 295461:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1782229531/real 1782229531] req@000000004dd37fdb x1868802794080000/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1782229547 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 9280.922520] LustreError: 305799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782229548 with bad export cookie 12261470476885070999 [ 9280.931202] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9280.934793] LustreError: 305799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9281.300229] Lustre: server umount lustre-MDT0001 complete [ 9292.481437] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 9295.642835] Key type lgssc unregistered [ 9295.973630] LNet: 307017:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9295.986749] LNetError: 307017:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9296.012239] LNet: Removed LNI 192.168.204.106@tcp [ 9296.827727] Key type .llcrypt unregistered [ 9296.829967] Key type ._llcrypt unregistered [ 9316.919877] Key type ._llcrypt registered [ 9316.925253] Key type .llcrypt registered [ 9317.113727] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 9332.275888] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9333.357635] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9333.492955] alg: No test for adler32 (adler32-zlib) [ 9334.601776] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [ 9334.886052] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 9336.536456] Key type lgssc registered [ 9337.558509] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9359.714584] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9373.901514] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9373.929032] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9375.232026] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9375.302415] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9375.413465] Lustre: lustre-MDT0000: new disk, initializing [ 9375.546575] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9375.570560] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9380.608810] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9390.618300] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9390.904571] Lustre: lustre-OST0000: new disk, initializing [ 9390.910226] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9390.918897] Lustre: Skipped 1 previous similar message [ 9390.926594] Lustre: 311064:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9391.014866] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9397.469172] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9398.381849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9398.393788] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9398.438381] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10273.272941] Lustre: server umount lustre-MDT0000 complete [10276.786947] LustreError: 310251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782230544 with bad export cookie 16981352638111748775 [10276.802287] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10287.275804] Lustre: server umount lustre-OST0000 complete [10313.466638] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [10324.669718] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [10363.379506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10363.621718] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10363.664337] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10363.759520] Lustre: lustre-MDT0000: new disk, initializing [10363.836670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10363.858191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10368.615180] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10378.992720] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10379.133679] Lustre: 315421:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10379.196085] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10379.200302] Lustre: Skipped 1 previous similar message [10379.295346] Lustre: lustre-MDT0001: new disk, initializing [10379.396911] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10379.429394] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10379.436965] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10384.155581] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10393.997705] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10400.243127] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10407.631099] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10407.958281] Lustre: lustre-OST0000: new disk, initializing [10407.964176] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10407.977413] Lustre: 317388:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10408.077235] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10414.924784] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10416.757325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10416.763527] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10416.886800] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10424.966987] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10430.397483] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10430.743297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10435.716600] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10436.026620] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10442.726336] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10442.732336] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10442.734558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10442.746157] Lustre: Skipped 1 previous similar message [10444.220374] Lustre: server umount lustre-OST0000 complete [10450.913126] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10450.919733] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10450.933659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10450.936233] Lustre: Skipped 1 previous similar message [10452.964896] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10452.967289] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10454.462716] Lustre: server umount lustre-MDT0000 complete [10458.394056] LustreError: 315412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782230726 with bad export cookie 16981352638114689587 [10458.401809] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10458.402986] LustreError: 315412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10458.946114] Lustre: server umount lustre-MDT0001 complete [10462.354202] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10464.194028] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 12:05:30 (1782230730) [10470.172213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10470.570420] LustreError: 319703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10470.583952] LustreError: 319703:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10470.632262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10475.431320] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10476.002912] LustreError: 319704:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10481.126065] LustreError: 319703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10483.633122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10483.997861] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10488.689484] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10496.576386] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10502.328937] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10508.748668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10509.119570] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10515.318117] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10522.776419] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10529.758593] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10529.943757] Lustre: lustre-OST0001: new disk, initializing [10529.950634] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10529.956931] Lustre: 322541:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10530.086814] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10536.478591] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10536.592510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10536.604527] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10536.655223] LustreError: 322902:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10537.701094] LustreError: 322902:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10537.714887] LustreError: 322902:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10538.789117] LustreError: 322902:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10538.804213] LustreError: 322902:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10540.843307] LustreError: 322909:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10540.858467] LustreError: 322909:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10545.061946] LustreError: 322909:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10545.072122] LustreError: 322909:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10546.904287] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10547.235779] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10552.541411] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b9090851800.ost_server_uuid 50 [10553.257657] LustreError: 322902:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10553.275112] LustreError: 322902:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10554.073355] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b9090851800.ost_server_uuid in FULL state after 0 sec [10559.092415] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8b9090851800.ost_server_uuid 50 [10560.523564] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8b9090851800.ost_server_uuid in FULL state after 0 sec [10562.609670] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10577.888718] Lustre: lustre-OST0001: Not available for connect from 192.168.204.6@tcp (stopping) [10577.890665] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10577.899017] Lustre: Skipped 1 previous similar message [10577.901667] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10577.915170] LustreError: Skipped 1 previous similar message [10577.946722] Lustre: Skipped 1 previous similar message [10581.448139] Lustre: server umount lustre-OST0001 complete [10582.997057] LustreError: 322454:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10588.130976] LustreError: 324148:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10588.149428] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10588.150533] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10588.152424] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10588.152431] Lustre: Skipped 2 previous similar messages [10588.156484] LustreError: 324148:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10593.106736] Lustre: server umount lustre-OST0000 complete [10598.373502] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10598.378663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10598.394096] Lustre: Skipped 2 previous similar messages [10598.399835] Lustre: Skipped 1 previous similar message [10603.504509] LustreError: 319703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10603.612707] Lustre: server umount lustre-MDT0000 complete [10607.136301] LustreError: 322540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782230874 with bad export cookie 16981352638114690469 [10607.139458] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10607.148201] LustreError: 322540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [10607.576297] Lustre: server umount lustre-MDT0001 complete [10618.733409] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [10621.567953] Key type lgssc unregistered [10621.861091] LNet: 325229:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10621.873945] LNetError: 325229:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10621.899152] LNet: Removed LNI 192.168.204.106@tcp [10622.814191] Key type .llcrypt unregistered [10622.816316] Key type ._llcrypt unregistered [10647.487690] Key type ._llcrypt registered [10647.490630] Key type .llcrypt registered [10647.640494] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [10648.999461] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10649.027833] alg: No test for adler32 (adler32-zlib) [10650.102266] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [10650.437956] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [10652.176188] Key type lgssc registered [10653.422151] Lustre: Echo OBD driver; http://www.lustre.org/ [10661.230450] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10661.249790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10662.652149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10667.212774] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10673.125967] LustreError: 326655:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10673.137379] LustreError: 326655:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10675.434835] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10678.252530] LustreError: 326654:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10681.445414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10681.726398] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10686.341699] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10696.393633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10696.854961] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10703.425579] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10707.122801] LustreError: 328335:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10708.236360] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10711.047581] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10713.575904] LustreError: 328335:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10713.604026] LustreError: 328335:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10718.708549] LustreError: 328336:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10718.727031] LustreError: 328336:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10718.895396] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10719.166288] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10724.354824] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10725.340262] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10732.341179] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10740.059350] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10740.273160] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10842.093327] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10842.107380] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10842.110546] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10842.118796] Lustre: Skipped 1 previous similar message [10842.136023] Lustre: Skipped 1 previous similar message [10847.209903] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10847.223410] Lustre: Skipped 1 previous similar message [10848.028770] Lustre: server umount lustre-OST0001 complete [10852.327100] LustreError: 329317:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10852.347702] LustreError: 329317:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10855.909241] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10855.917056] LustreError: Skipped 1 previous similar message [10855.920692] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10855.935711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10856.935949] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10856.953451] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10859.794060] Lustre: server umount lustre-OST0000 complete [10866.147257] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10866.165903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10866.184193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10866.193372] Lustre: Skipped 3 previous similar messages [10870.002500] Lustre: server umount lustre-MDT0000 complete [10872.803348] LustreError: 330342:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10872.824338] LustreError: 330342:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10873.626061] LustreError: 330341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782231141 with bad export cookie 4556383249328844278 [10873.631213] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10873.640980] LustreError: 330341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10874.122856] Lustre: server umount lustre-MDT0001 complete [10886.039573] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [10889.417673] Key type lgssc unregistered [10889.814605] LNet: 331599:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10889.827704] LNetError: 331599:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10889.850115] LNet: Removed LNI 192.168.204.106@tcp [10890.707595] Key type .llcrypt unregistered [10890.709811] Key type ._llcrypt unregistered [10938.730339] Key type ._llcrypt registered [10938.734760] Key type .llcrypt registered [10938.853237] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [10939.770080] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10939.805730] alg: No test for adler32 (adler32-zlib) [10941.155476] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [10941.410827] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [10943.080182] Key type lgssc registered [10944.233929] Lustre: Echo OBD driver; http://www.lustre.org/ [10955.389668] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10955.433743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10957.073624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10962.252525] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10966.507490] LustreError: 333890:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10966.519224] LustreError: 333890:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10970.917378] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10971.403921] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10976.347661] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10979.475609] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10986.070424] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10986.485885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10993.413976] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10996.717095] LustreError: 335382:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11001.830204] LustreError: 335382:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11001.843648] LustreError: 335382:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11001.857162] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11002.472247] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11002.915345] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11008.000993] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11009.766266] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11019.487270] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11026.652738] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11043.299751] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11043.315305] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11043.336305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11043.825752] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11043.854334] Lustre: Skipped 2 previous similar messages [11043.860239] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11043.864514] Lustre: Skipped 2 previous similar messages [11045.532358] Lustre: server umount lustre-MDT0000 complete [11048.937311] LustreError: 333890:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11048.953361] LustreError: 333890:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11049.310105] LustreError: 333872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782231317 with bad export cookie 16378865287037267066 [11049.325366] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11049.331350] LustreError: 333872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11049.754499] Lustre: server umount lustre-MDT0001 complete [11064.463280] Lustre: server umount lustre-OST0000 complete [11079.154422] Lustre: server umount lustre-OST0001 complete [11097.840282] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [11108.793830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11109.263783] LustreError: 339155:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11109.370711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11114.648578] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11118.628222] LustreError: 339155:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11118.650300] LustreError: 339155:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11124.477099] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11125.042536] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11129.619782] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11132.789783] Lustre: 340295:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11139.823645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11147.921574] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11149.676518] LustreError: 340651:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11149.706168] LustreError: 340651:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11153.787772] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11158.107717] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11158.487156] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11158.509468] Lustre: Skipped 1 previous similar message [11163.660751] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11165.711410] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11173.676197] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11177.471768] Lustre: 342165:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11181.537731] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11181.544644] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11181.583571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11184.102805] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11184.103466] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11184.128394] Lustre: Skipped 2 previous similar messages [11184.133828] Lustre: Skipped 2 previous similar messages [11187.637811] Lustre: server umount lustre-MDT0000 complete [11189.220374] LustreError: 339156:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11189.242542] LustreError: 339156:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [11191.122092] LustreError: 339135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782231458 with bad export cookie 16378865287037268767 [11191.136530] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11191.147294] LustreError: 339135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11191.480390] Lustre: server umount lustre-MDT0001 complete [11205.414829] Lustre: server umount lustre-OST0000 complete [11218.844303] Lustre: server umount lustre-OST0001 complete [11222.778095] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11224.781399] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11226.467130] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:18:12 (1782231492) [11249.883528] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [11259.466070] loop0: detected capacity change from 0 to 1099511627776 [11260.218138] SGI XFS with ACLs, security attributes, quota, no debug enabled [11260.223936] *** VALIDATE xfs *** [11260.232398] XFS (loop0): Mounting V5 Filesystem [11260.266313] XFS (loop0): Ending clean mount [11261.651371] loop1: detected capacity change from 0 to 18691697672192 [11393.455734] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [11405.971917] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [11444.536819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11444.803568] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11444.859814] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11444.971613] Lustre: lustre-MDT0000: new disk, initializing [11445.044971] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11445.072636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11449.911994] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11462.384957] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11462.559421] Lustre: 347307:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11462.656486] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11462.663997] Lustre: Skipped 1 previous similar message [11462.806923] Lustre: lustre-MDT0001: new disk, initializing [11463.130423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11463.147055] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11468.810335] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11479.900132] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11486.453349] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11494.591520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11495.105569] Lustre: lustre-OST0000: new disk, initializing [11495.109307] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11495.136883] Lustre: 349282:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11496.234920] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11496.247253] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11496.350058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11502.373685] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11513.227671] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11518.093543] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11518.328722] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11522.658867] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11522.949162] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11527.144682] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11527.158892] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11527.160996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11527.180946] Lustre: Skipped 2 previous similar messages [11531.162046] Lustre: server umount lustre-OST0000 complete [11534.817935] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11537.379191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11537.380630] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11537.392575] Lustre: Skipped 1 previous similar message [11537.414847] Lustre: Skipped 2 previous similar messages [11549.152190] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11549.371524] Lustre: server umount lustre-MDT0000 complete [11551.718746] LustreError: 347315:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11551.752233] LustreError: 347315:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11552.693362] LustreError: 347301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782231820 with bad export cookie 16378865287037270321 [11552.702912] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11552.712250] LustreError: 347301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11553.073880] Lustre: server umount lustre-MDT0001 complete [11560.498414] XFS (loop0): Unmounting Filesystem [11565.427698] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:23:51 (1782231831) [11572.483570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11573.010175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11573.021798] Lustre: Skipped 2 previous similar messages [11577.086543] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11585.629330] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11591.343916] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11599.908627] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11605.789562] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11613.215290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11619.970687] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11628.046653] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11641.313521] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11641.323935] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11641.329233] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11641.340518] Lustre: Skipped 1 previous similar message [11641.346279] Lustre: Skipped 5 previous similar messages [11646.539885] Lustre: server umount lustre-OST0000 complete [11664.352401] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11664.520886] Lustre: server umount lustre-MDT0000 complete [11668.014295] LustreError: 351822:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782231935 with bad export cookie 16378865287037271203 [11668.015698] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11668.022304] LustreError: 351822:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11668.380470] Lustre: server umount lustre-MDT0001 complete [11678.690252] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [11681.659484] Key type lgssc unregistered [11682.061823] LNet: 355472:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11682.071692] LNetError: 355472:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11682.085878] LNet: Removed LNI 192.168.204.106@tcp [11682.951284] Key type .llcrypt unregistered [11682.953194] Key type ._llcrypt unregistered [11711.552165] Key type ._llcrypt registered [11711.558828] Key type .llcrypt registered [11711.719885] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [11712.785180] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11712.896768] alg: No test for adler32 (adler32-zlib) [11713.967942] Lustre: Lustre: Build Version: 2.17.54_83_gf3505cf [11714.245081] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [11715.920212] Key type lgssc registered [11717.099786] Lustre: Echo OBD driver; http://www.lustre.org/ [11724.537663] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11724.569078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11726.052217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11729.829991] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11735.528059] LustreError: 357044:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11735.551805] LustreError: 357044:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11737.233560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11737.588807] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11741.705618] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11748.845782] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11754.132023] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11760.663532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11761.054611] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11767.112605] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11774.181297] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11781.606053] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11781.630485] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11781.636863] Lustre: Skipped 1 previous similar message [11786.724860] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11786.734758] Lustre: Skipped 2 previous similar messages [11787.213459] LustreError: 357039:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11787.288484] Lustre: server umount lustre-MDT0001 complete [11791.842147] LustreError: 357043:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11791.866396] LustreError: 357043:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11796.961545] LustreError: 357044:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11796.986860] LustreError: 357044:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11797.035483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11797.190466] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11797.194298] Lustre: Found index 1 for lustre-MDT0001, updating log [11797.254273] Lustre: 357031:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11797.509778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11801.813591] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [11802.609929] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11807.249379] LustreError: 360642:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11807.254807] LustreError: 360642:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11807.270555] LustreError: 357037:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11807.347469] LustreError: 360645:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -17 [11807.359899] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11807.359899] [12109.783660] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [12109.794876] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12109.800907] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12109.832417] Lustre: Skipped 2 previous similar messages [12114.878151] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [12114.886043] Lustre: Skipped 2 previous similar messages [12119.999777] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [12120.007633] Lustre: Skipped 2 previous similar messages [12120.032150] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12120.192874] Lustre: server umount lustre-MDT0001 complete [12125.123747] LustreError: 364984:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12128.313339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12128.416725] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12128.423728] Lustre: Found index 1 for lustre-MDT0001, updating log [12128.464751] Lustre: 357033:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12128.786216] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12132.032533] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [12133.865899] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12134.904921] LustreError: 369818:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12134.911936] LustreError: 369818:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12134.919538] LustreError: 357037:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12134.926375] LustreError: 357037:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12134.968563] LustreError: 369821:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12134.977234] LustreError: 369821:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12134.982589] LustreError: 369821:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -17 [12134.998863] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.106@tcp [12134.998863] [12437.460712] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [12437.466815] Lustre: Skipped 2 previous similar messages [12439.012659] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12443.790182] Lustre: server umount lustre-MDT0001 complete [12444.136442] LustreError: 364984:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12444.157508] LustreError: 364984:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12456.056119] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12456.147678] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12456.151288] Lustre: Found index 1 for lustre-MDT0001, updating log [12456.175665] Lustre: 357031:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12456.687924] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12462.112298] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12462.318521] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [12464.630653] LustreError: 379154:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12464.640880] LustreError: 379154:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12464.647569] LustreError: 357037:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12464.658297] LustreError: 357037:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12464.698381] LustreError: 379161:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12464.711035] LustreError: 379161:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -114 [12464.715540] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12464.715540] [12784.610935] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12784.611958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12784.614406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12784.614414] Lustre: Skipped 3 previous similar messages [12784.630654] Lustre: Skipped 2 previous similar messages [12790.760915] Lustre: server umount lustre-MDT0000 complete [12794.851241] LustreError: 359427:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12794.869554] LustreError: 359427:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [12795.585962] LustreError: 369404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782233063 with bad export cookie 5548389598996560315 [12795.595880] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12799.971547] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12799.992520] Lustre: Skipped 1 previous similar message [12802.015853] Lustre: server umount lustre-MDT0001 complete [12806.769138] Lustre: server umount lustre-OST0000 complete [12817.046269] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [12828.467593] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [12866.641274] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:45:32 (1782233132) [12873.955383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12874.261048] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12874.307986] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12874.413854] Lustre: lustre-MDT0000: new disk, initializing [12874.522111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12874.549765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12879.414128] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [12890.026601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12890.123949] Lustre: 389775:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12890.171568] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12890.175068] Lustre: Skipped 1 previous similar message [12890.235228] Lustre: lustre-MDT0001: new disk, initializing [12890.357238] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12890.419961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12890.431735] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12895.922581] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [12906.905559] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12913.290122] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12921.133993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12921.543590] Lustre: lustre-OST0000: new disk, initializing [12921.554210] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12921.570236] Lustre: 391742:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12921.739992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12927.114165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12927.129339] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12927.289921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12929.148472] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [12939.926701] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12947.948109] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12947.956520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12947.971761] Lustre: Skipped 1 previous similar message [12947.993437] Lustre: Skipped 9 previous similar messages [12950.350205] Lustre: server umount lustre-MDT0000 complete [12953.059391] LustreError: 392217:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12953.090334] LustreError: 392217:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [12955.005862] LustreError: 390569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1782233222 with bad export cookie 5548389598996562268 [12955.006937] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12955.014951] LustreError: 390581:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12955.024158] LustreError: 390569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12955.671251] Lustre: server umount lustre-MDT0001 complete [12971.078655] Lustre: server umount lustre-OST0000 complete [13006.210772] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [13018.295987] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [13056.358803] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12789 sec ========= 12:48:42 (1782233322) [13058.431095] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:48:44 (1782233324) === [13063.139908] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:48:49 (1782233329) === [13109.954208] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [13113.154885] Key type lgssc unregistered [13113.488961] LNet: 398111:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13113.499480] LNetError: 398111:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [13113.515883] LNet: Removed LNI 192.168.204.106@tcp [13114.480608] Key type .llcrypt unregistered [13114.484336] Key type ._llcrypt unregistered