[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 499892436 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2512MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002363] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004017] kvm-guest: setup PV IPIs [ 0.007754] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.011008] pid_max: default: 32768 minimum: 301 [ 0.012161] LSM: Security Framework initializing [ 0.013050] Yama: becoming mindful. [ 0.014036] SELinux: Initializing. [ 0.015071] *** VALIDATE selinux *** [ 0.023221] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027805] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029045] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030103] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031112] *** VALIDATE tmpfs *** [ 0.033067] *** VALIDATE proc *** [ 0.034259] *** VALIDATE cgroup *** [ 0.035012] *** VALIDATE cgroup2 *** [ 0.036245] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037171] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039030] Spectre V2 : User space: Vulnerable [ 0.040010] Speculative Store Bypass: Vulnerable [ 0.042870] debug: unmapping init [mem 0xffffffff9e059000-0xffffffff9e060fff] [ 0.044176] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045578] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046027] ... version: 2 [ 0.047014] ... bit width: 48 [ 0.048014] ... generic registers: 4 [ 0.049013] ... value mask: 0000ffffffffffff [ 0.050018] ... max period: 00007fffffffffff [ 0.051017] ... fixed-purpose events: 3 [ 0.052013] ... event mask: 000000070000000f [ 0.053336] rcu: Hierarchical SRCU implementation. [ 0.055371] smp: Bringing up secondary CPUs ... [ 0.056605] x86: Booting SMP configuration: [ 0.057023] .... node #0, CPUs: #1 #2 #3 [ 0.060461] smp: Brought up 1 node, 4 CPUs [ 0.062014] smpboot: Max logical packages: 1 [ 0.063019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.149505] node 0 deferred pages initialised in 82ms [ 0.155137] devtmpfs: initialized [ 0.157240] x86/mm: Memory block size: 128MB [ 0.159813] gcov: version magic: 0x41383552 [ 0.161200] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.163121] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.166320] pinctrl core: initialized pinctrl subsystem [ 0.168210] [ 0.168761] ************************************************************* [ 0.171014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.173015] ** ** [ 0.175013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.177013] ** ** [ 0.179012] ** This means that this kernel is built to expose internal ** [ 0.182013] ** IOMMU data structures, which may compromise security on ** [ 0.184012] ** your system. ** [ 0.186014] ** ** [ 0.189015] ** If you see this message and you are not debugging the ** [ 0.191011] ** kernel, report this immediately to your vendor! ** [ 0.193020] ** ** [ 0.196012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.198013] ************************************************************* [ 0.199671] NET: Registered protocol family 16 [ 0.202545] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.205055] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.208065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.211104] cpuidle: using governor menu [ 0.213000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.215535] PCI: Using configuration type 1 for base access [ 0.217123] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.228085] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.231021] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.235079] cryptd: max_cpu_qlen set to 1000 [ 0.239286] ACPI: Added _OSI(Module Device) [ 0.240025] ACPI: Added _OSI(Processor Device) [ 0.241000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.242017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.247109] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.254694] ACPI: Interpreter enabled [ 0.256066] ACPI: PM: (supports S0 S3 S4 S5) [ 0.257014] ACPI: Using IOAPIC for interrupt routing [ 0.259103] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.263406] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.272577] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.275058] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.279024] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.282075] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.287286] acpiphp: Slot [2] registered [ 0.288074] acpiphp: Slot [5] registered [ 0.290131] acpiphp: Slot [6] registered [ 0.292167] acpiphp: Slot [7] registered [ 0.293109] acpiphp: Slot [8] registered [ 0.295110] acpiphp: Slot [9] registered [ 0.296117] acpiphp: Slot [10] registered [ 0.298117] acpiphp: Slot [3] registered [ 0.299093] acpiphp: Slot [4] registered [ 0.300125] acpiphp: Slot [11] registered [ 0.302087] acpiphp: Slot [12] registered [ 0.303089] acpiphp: Slot [13] registered [ 0.304115] acpiphp: Slot [14] registered [ 0.306090] acpiphp: Slot [15] registered [ 0.307104] acpiphp: Slot [16] registered [ 0.309085] acpiphp: Slot [17] registered [ 0.311107] acpiphp: Slot [18] registered [ 0.312135] acpiphp: Slot [19] registered [ 0.314302] acpiphp: Slot [20] registered [ 0.316107] acpiphp: Slot [21] registered [ 0.317126] acpiphp: Slot [22] registered [ 0.319106] acpiphp: Slot [23] registered [ 0.320097] acpiphp: Slot [24] registered [ 0.322131] acpiphp: Slot [25] registered [ 0.323107] acpiphp: Slot [26] registered [ 0.325109] acpiphp: Slot [27] registered [ 0.326100] acpiphp: Slot [28] registered [ 0.328097] acpiphp: Slot [29] registered [ 0.329119] acpiphp: Slot [30] registered [ 0.331182] acpiphp: Slot [31] registered [ 0.332037] PCI host bridge to bus 0000:00 [ 0.334031] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.336021] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.338025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.341025] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.344024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.347027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.349174] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.352071] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.355300] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.365702] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.371595] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.374019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.376020] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.379019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.382386] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.385814] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.388042] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.391821] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.397016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.410017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.418014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.423586] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.432018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.442016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.459018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.470832] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.483017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.492016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.513016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.522543] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.536019] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.548018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.562081] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.570892] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.579016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.587017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.604017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.612000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.624015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.631017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.645017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.655184] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.664015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.674020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.690016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.701394] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.704319] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.707375] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.709319] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.712216] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.717113] iommu: Default domain type: Passthrough [ 0.719491] SCSI subsystem initialized [ 0.721227] ACPI: bus type USB registered [ 0.723096] usbcore: registered new interface driver usbfs [ 0.725099] usbcore: registered new interface driver hub [ 0.727085] usbcore: registered new device driver usb [ 0.729175] pps_core: LinuxPPS API ver. 1 registered [ 0.731013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.735063] PTP clock support registered [ 0.737132] EDAC MC: Ver: 3.0.0 [ 0.740176] PCI: Using ACPI for IRQ routing [ 0.741508] NetLabel: Initializing [ 0.743014] NetLabel: domain hash size = 128 [ 0.744010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.746075] NetLabel: unlabeled traffic allowed by default [ 0.749042] vgaarb: loaded [ 0.750233] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.752016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.757888] clocksource: Switched to clocksource kvm-clock [ 0.858622] VFS: Disk quotas dquot_6.6.0 [ 0.860219] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.862752] *** VALIDATE ramfs *** [ 0.863881] *** VALIDATE hugetlbfs *** [ 0.865549] pnp: PnP ACPI init [ 0.868065] pnp: PnP ACPI: found 6 devices [ 0.887306] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.890747] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.892926] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.895162] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.897631] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.899816] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.902735] NET: Registered protocol family 2 [ 0.905065] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.909684] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.912845] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.917858] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.921477] TCP: Hash tables configured (established 65536 bind 65536) [ 0.924146] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.927228] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.930080] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.933125] NET: Registered protocol family 1 [ 0.935849] RPC: Registered named UNIX socket transport module. [ 0.938023] RPC: Registered udp transport module. [ 0.939686] RPC: Registered tcp transport module. [ 0.941397] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.943667] NET: Registered protocol family 44 [ 0.945190] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.947182] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.948748] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.950516] PCI: CLS 0 bytes, default 64 [ 0.952164] Unpacking initramfs... [ 2.328899] debug: unmapping init [mem 0xffff963bfcc54000-0xffff963bfffbffff] [ 2.332522] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.335037] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.338012] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.823632] Initialise system trusted keyrings [ 2.825341] Key type blacklist registered [ 2.827685] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.838297] zbud: loaded [ 2.841515] *** VALIDATE nfs *** [ 2.842883] *** VALIDATE nfs4 *** [ 2.844722] pstore: using deflate compression [ 2.848581] Platform Keyring initialized [ 2.940804] NET: Registered protocol family 38 [ 2.942027] Key type asymmetric registered [ 2.943143] Asymmetric key parser 'x509' registered [ 2.944494] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.947437] io scheduler mq-deadline registered [ 2.948700] io scheduler kyber registered [ 2.950107] io scheduler bfq registered [ 2.951367] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.953397] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.956300] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.959304] ACPI: Power Button [PWRF] [ 2.964224] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.970941] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.983174] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.997094] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.010185] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.038135] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.065565] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.070292] Non-volatile memory driver v1.3 [ 3.071839] Linux agpgart interface v0.103 [ 3.100611] virtio_blk virtio1: [vda] 150616 512-byte logical blocks (77.1 MB/73.5 MiB) [ 3.103434] vda: detected capacity change from 0 to 77115392 [ 3.117659] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.119848] vdb: detected capacity change from 0 to 1073741824 [ 3.133471] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.135820] vdc: detected capacity change from 0 to 2621440000 [ 3.154566] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.157388] vdd: detected capacity change from 0 to 2621440000 [ 3.178185] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.180578] vde: detected capacity change from 0 to 4294967296 [ 3.205742] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.208404] vdf: detected capacity change from 0 to 4294967296 [ 3.224115] libphy: Fixed MDIO Bus: probed [ 3.230503] usbcore: registered new interface driver usbserial_generic [ 3.232374] usbserial: USB Serial support registered for generic [ 3.234490] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.238377] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.239867] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.243066] mousedev: PS/2 mouse device common for all mice [ 3.245977] rtc_cmos 00:05: RTC can wake from S4 [ 3.250189] rtc_cmos 00:05: registered as rtc0 [ 3.251670] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.252553] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.256612] intel_pstate: CPU model not supported [ 3.258487] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.259345] hid: raw HID events driver (C) Jiri Kosina [ 3.262296] usbcore: registered new interface driver usbhid [ 3.262339] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.263510] usbhid: USB HID core driver [ 3.263636] drop_monitor: Initializing network drop monitor service [ 3.268108] Initializing XFRM netlink socket [ 3.269978] NET: Registered protocol family 10 [ 3.272248] Segment Routing with IPv6 [ 3.273512] NET: Registered protocol family 17 [ 3.275028] mpls_gso: MPLS GSO support [ 3.278621] RAS: Correctable Errors collector initialized. [ 3.280040] AVX version of gcm_enc/dec engaged. [ 3.281158] AES CTR mode by8 optimization enabled [ 3.350511] sched_clock: Marking stable (3350491882, 0)->(4233961983, -883470101) [ 3.353513] registered taskstats version 1 [ 3.355575] Loading compiled-in X.509 certificates [ 3.356899] zswap: loaded using pool lzo/zbud [ 3.381261] Key type big_key registered [ 3.391879] Key type encrypted registered [ 3.393828] ima: No TPM chip found, activating TPM-bypass! [ 3.395972] ima: Allocated hash algorithm: sha1 [ 3.397656] ima: No architecture policies found [ 3.399218] evm: Initialising EVM extended attributes: [ 3.400840] evm: security.selinux [ 3.402078] evm: security.ima [ 3.403072] evm: security.capability [ 3.404356] evm: HMAC attrs: 0x1 [ 3.406610] rtc_cmos 00:05: setting system clock to 2026-09-30 14:46:40 UTC (1790779600) [ 3.412162] debug: unmapping init [mem 0xffffffff9f003000-0xffffffff9f1fffff] [ 3.415025] debug: unmapping init [mem 0xffffffff9dd82000-0xffffffff9e058fff] [ 3.425589] Write protecting the kernel read-only data: 28672k [ 3.428053] debug: unmapping init [mem 0xffffffff9c403000-0xffffffff9c5fffff] [ 3.431264] debug: unmapping init [mem 0xffffffff9cd14000-0xffffffff9cdfffff] [ 3.461742] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.469326] systemd[1]: Detected virtualization kvm. [ 3.471332] systemd[1]: Detected architecture x86-64. [ 3.473283] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.502135] systemd[1]: No hostname configured. [ 3.503771] systemd[1]: Set hostname to . [ 3.505769] random: systemd: uninitialized urandom read (16 bytes read) [ 3.507917] systemd[1]: Initializing machine ID from random generator. [ 3.626508] random: systemd: uninitialized urandom read (16 bytes read) [ 3.629107] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.632327] random: systemd: uninitialized urandom read (16 bytes read) [ 3.634786] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.638641] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.196096] device-mapper: uevent: version 1.0.3 [ 4.198820] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.826818] random: fast init done [ 4.840472] virtio_net virtio0 ens2: renamed from eth0 [ 4.889493] scsi host0: ata_piix [ 4.940377] scsi host1: ata_piix [ 4.951429] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.954317] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.528834] dracut-initqueue[577]: RTNETLINK answers: File exists [ 9.910227] random: crng init done [ 9.911720] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.118252] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.156357] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.367534] SELinux: Disabled at runtime. [ 11.428513] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.437076] systemd[1]: Detected virtualization kvm. [ 11.438465] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.881612] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.885114] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.889020] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.892390] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.895901] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.903924] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.913223] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ 11.990995] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.295507] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.512633] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.603446] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.684758] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.702640] EDAC sbridge: Ver: 1.1.2 [ 14.224905] Key type dns_resolver registered [ 14.516208] NFS: Registering the id_resolver key type [ 14.518128] Key type id_resolver registered [ 14.519390] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg141-server login: [ 38.942202] libcfs: loading out-of-tree module taints kernel. [ 38.962645] Key type ._llcrypt registered [ 38.964272] Key type .llcrypt registered [ 39.009516] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 50.207698] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 51.966690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 51.987227] alg: No test for adler32 (adler32-zlib) [ 53.539835] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 54.785633] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 56.583347] Key type lgssc registered [ 58.676456] Lustre: Echo OBD driver; http://www.lustre.org/ [ 77.256042] hrtimer: interrupt took 7550443 ns [ 79.339815] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 131.324412] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 146.433597] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 146.482598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 147.877549] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 147.944622] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 148.086509] Lustre: lustre-MDT0000: new disk, initializing [ 148.190048] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 148.218927] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 153.448263] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 166.984096] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 167.066492] Lustre: 6483:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 167.098443] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 167.102041] Lustre: Skipped 1 previous similar message [ 167.185464] Lustre: lustre-MDT0001: new disk, initializing [ 167.287496] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 167.322153] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 167.341757] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 171.710878] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 177.854976] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 189.472545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 189.786201] Lustre: lustre-OST0000: new disk, initializing [ 189.791383] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 189.800595] Lustre: 8419:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 189.875693] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 190.258482] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 190.280221] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 190.332056] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 197.440327] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 212.779784] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 212.970625] Lustre: lustre-OST0001: new disk, initializing [ 212.977667] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 212.988550] Lustre: 9492:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 213.151159] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 219.729925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 219.734110] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 219.809882] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 221.832248] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 235.978337] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 244.151383] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 251.748837] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing check_logdir /tmp/testlogs/ [ 258.801509] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing yml_node [ 264.173847] Lustre: DEBUG MARKER: Client: 2.17.59.2 [ 267.486473] Lustre: DEBUG MARKER: MDS: 2.17.59.2 [ 271.257871] Lustre: DEBUG MARKER: OSS: 2.17.59.2 [ 273.452739] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Sep 30 10:51:08 EDT 2026 [ 293.317691] Lustre: DEBUG MARKER: only running tests: 76-120 [ 295.294852] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 297.325756] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 315.875256] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 315.886393] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 315.907348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.411541] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.412351] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 317.419852] Lustre: Skipped 1 previous similar message [ 317.434343] Lustre: Skipped 1 previous similar message [ 321.123372] Lustre: server umount lustre-MDT0000 complete [ 325.420417] LustreError: 7429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779922 with bad export cookie 12524111475575122326 [ 325.444458] LustreError: 7429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 325.456223] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 327.651549] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 327.658412] LustreError: 8901:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 327.682719] Lustre: Skipped 1 previous similar message [ 327.694708] LustreError: 8901:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 327.730968] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 327.741904] Lustre: Skipped 1 previous similar message [ 330.656106] LustreError: 8901:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.657381] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 330.667177] LustreError: 8901:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 330.693166] Lustre: Skipped 1 previous similar message [ 332.325523] Lustre: server umount lustre-MDT0001 complete [ 337.401468] Lustre: server umount lustre-OST0000 complete [ 343.103703] Lustre: server umount lustre-OST0001 complete [ 353.772451] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 366.594192] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 413.123786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 413.634318] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 413.697650] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 413.818928] Lustre: lustre-MDT0000: new disk, initializing [ 413.991340] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 414.022937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 419.106384] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 430.844120] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 431.065839] Lustre: 15100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 431.232402] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 431.240308] Lustre: Skipped 1 previous similar message [ 431.357835] Lustre: lustre-MDT0001: new disk, initializing [ 431.603098] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 431.692282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 431.722448] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 437.345850] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 450.520907] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 459.045757] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 468.343668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 468.934335] Lustre: lustre-OST0000: new disk, initializing [ 468.943621] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 468.975527] Lustre: 17072:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 469.083184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 470.735723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 470.750625] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 470.920182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 476.620095] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 487.896645] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 493.205424] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 493.381513] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 498.774409] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 498.969462] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 501.730896] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 501.735854] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 501.742632] LustreError: Skipped 1 previous similar message [ 501.743849] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 501.776382] Lustre: Skipped 2 previous similar messages [ 507.304959] Lustre: server umount lustre-OST0000 complete [ 511.969195] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 511.991185] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 511.993870] Lustre: Skipped 3 previous similar messages [ 526.305064] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 526.685437] Lustre: server umount lustre-MDT0000 complete [ 527.334246] LustreError: 15112:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 527.355661] LustreError: 15112:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 530.700712] LustreError: 15093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780127 with bad export cookie 12524111475575123859 [ 530.706708] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 530.712257] LustreError: 15093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 531.406049] Lustre: server umount lustre-MDT0001 complete [ 537.464342] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:55:32 (1790780132) [ 544.342085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 544.945378] LustreError: 19339:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 545.055588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 550.369690] LustreError: 19340:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 550.393380] LustreError: 19340:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 550.583946] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 560.617759] LustreError: 19339:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 560.640892] LustreError: 19339:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 561.502059] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 567.006965] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 575.836969] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 583.430635] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 591.528384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 598.660874] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 608.975779] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 624.963196] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 633.845140] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 633.861979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 633.888029] Lustre: Skipped 1 previous similar message [ 633.895531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 633.898681] Lustre: Skipped 4 previous similar messages [ 638.063704] Lustre: server umount lustre-MDT0000 complete [ 641.507427] LustreError: 19340:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 641.522039] LustreError: 19340:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 641.635421] LustreError: 21183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780238 with bad export cookie 12524111475575124741 [ 641.652139] LustreError: 21183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 641.658987] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 641.989960] Lustre: server umount lustre-MDT0001 complete [ 656.337806] Lustre: server umount lustre-OST0000 complete [ 682.774875] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 695.004782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 695.502524] LustreError: 23747:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 695.563454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 695.574664] Lustre: Skipped 2 previous similar messages [ 700.367987] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 709.719767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 714.853574] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 718.025254] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 725.669025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 732.922467] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 743.244341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 743.592387] Lustre: lustre-OST0001: new disk, initializing [ 743.598490] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 743.606883] Lustre: 26061:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 745.370689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 745.397549] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 745.567561] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 752.800325] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 764.256080] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 767.950176] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 781.283094] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 781.288467] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 781.288467] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 781.288476] Lustre: Skipped 3 previous similar messages [ 781.300270] Lustre: Skipped 4 previous similar messages [ 781.326115] Lustre: Skipped 2 previous similar messages [ 785.115948] Lustre: server umount lustre-MDT0000 complete [ 786.402559] LustreError: 23744:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 786.423689] LustreError: 23744:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 789.005795] LustreError: 23727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780386 with bad export cookie 12524111475575125588 [ 789.008973] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 789.038399] LustreError: 23727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 789.588716] Lustre: server umount lustre-MDT0001 complete [ 803.007089] Lustre: server umount lustre-OST0000 complete [ 817.758724] Lustre: server umount lustre-OST0001 complete [ 836.612872] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 846.684428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 847.245551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 847.254035] Lustre: Skipped 3 previous similar messages [ 851.827241] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 859.871273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 865.241173] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 867.938779] Lustre: 30242:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 875.099827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 882.052504] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 890.536849] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 897.262881] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 905.133419] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 909.607410] Lustre: 32123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 916.976759] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 916.993798] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 917.012563] Lustre: Skipped 1 previous similar message [ 917.028547] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 922.351800] Lustre: server umount lustre-MDT0000 complete [ 924.134529] LustreError: 29076:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 924.156412] LustreError: 29076:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 926.897784] LustreError: 29064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780523 with bad export cookie 12524111475575126883 [ 926.902500] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 926.919928] LustreError: 29064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 927.502382] Lustre: server umount lustre-MDT0001 complete [ 941.291753] Lustre: server umount lustre-OST0000 complete [ 954.321661] Lustre: server umount lustre-OST0001 complete [ 962.524334] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 11:02:38 (1790780558) [ 993.143171] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1003.523668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1009.330959] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1018.491111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1023.234851] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1026.514847] Lustre: 36078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1034.057316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1042.546813] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1052.275713] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1059.109583] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1067.421332] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1071.983840] Lustre: 37955:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1080.289715] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1080.301687] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1080.313329] Lustre: Skipped 3 previous similar messages [ 1083.287586] Lustre: server umount lustre-MDT0000 complete [ 1086.982843] LustreError: 34912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780684 with bad export cookie 12524111475575127982 [ 1086.992742] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1087.358937] Lustre: server umount lustre-MDT0001 complete [ 1101.895978] Lustre: server umount lustre-OST0000 complete [ 1115.938844] Lustre: server umount lustre-OST0001 complete [ 1124.606573] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 11:05:19 (1790780719) [ 1142.187575] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1153.227572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1153.624392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1153.643835] Lustre: Skipped 7 previous similar messages [ 1157.940632] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1167.567387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1173.111323] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1176.245719] Lustre: 41245:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1182.818180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1189.634808] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1192.502039] LustreError: 41600:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1192.517887] LustreError: 41600:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 14 previous similar messages [ 1197.812778] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1204.399700] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1212.463579] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1216.422275] Lustre: 43123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1217.498901] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1217.513512] Lustre: Skipped 1 previous similar message [ 1222.685766] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.41@tcp (stopping) [ 1222.689992] Lustre: Skipped 9 previous similar messages [ 1224.164451] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1224.172378] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1224.186442] Lustre: Skipped 3 previous similar messages [ 1227.225146] Lustre: server umount lustre-MDT0000 complete [ 1231.489505] LustreError: 40078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780828 with bad export cookie 12524111475575129088 [ 1231.501536] LustreError: 40078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1231.518182] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1237.951923] Lustre: server umount lustre-MDT0001 complete [ 1246.652304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1256.416975] LustreError: 3611:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff963c7ee4dc00 x1877768563744128/t0(0) o250->MGC192.168.201.141@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1261.441168] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1266.158859] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1270.008095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1275.126657] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1275.889606] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1275.895726] Lustre: Skipped 1 previous similar message [ 1283.149559] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1284.837759] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1291.518922] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1293.349561] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1300.968664] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1307.658474] Lustre: server umount lustre-MDT0000 complete [ 1311.489742] LustreError: 40077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780908 with bad export cookie 12524111475575130222 [ 1311.504097] LustreError: 40077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1311.508492] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1311.867448] Lustre: server umount lustre-MDT0001 complete [ 1318.586875] Lustre: server umount lustre-OST0000 complete [ 1322.388533] Lustre: server umount lustre-OST0001 complete [ 1330.525728] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 11:08:46 (1790780926) [ 1345.709556] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1355.370684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1360.257830] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1369.990627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1375.710195] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1378.564610] Lustre: 48886:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1384.694712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1391.515814] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1399.708357] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1406.688329] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1415.842962] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1421.690223] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1437.195949] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1437.292785] Lustre: server umount lustre-MDT0000 complete [ 1441.652602] LustreError: 47715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790781038 with bad export cookie 12524111475575131013 [ 1441.653143] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1441.673582] LustreError: 47715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1442.232286] Lustre: server umount lustre-MDT0001 complete [ 1457.531378] Lustre: server umount lustre-OST0000 complete [ 1458.400080] Lustre: 3612:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790781039/real 1790781039] req@ffff963b42967800 x1877768563834112/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790781055 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1461.400498] Lustre: server umount lustre-OST0001 complete [ 1472.366482] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 11:11:07 (1790781067) [ 1475.091946] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1477.325329] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 11:11:12 (1790781072) [ 1490.976208] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 1501.637915] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1527.540441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1527.958229] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1528.093288] Lustre: lustre-MDT0000: new disk, initializing [ 1528.291258] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1534.602596] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1547.256618] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1554.486894] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1554.896731] Lustre: lustre-OST0000: new disk, initializing [ 1554.900436] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1554.907475] Lustre: Skipped 1 previous similar message [ 1554.912702] Lustre: 54964:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1556.428808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1556.471520] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1556.521018] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1561.245947] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1571.574332] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1574.029725] Lustre: DEBUG MARKER: create test files [ 1607.651293] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1607.656588] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1607.672163] Lustre: Skipped 16 previous similar messages [ 1611.141380] Lustre: server umount lustre-OST0000 complete [ 1615.751905] Lustre: server umount lustre-MDT0000 complete [ 1628.075949] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1631.050685] Key type lgssc unregistered [ 1631.285082] LNet: 56595:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1631.289562] LNetError: 56595:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1631.300855] LNet: Removed LNI 192.168.201.141@tcp [ 1632.212351] Key type .llcrypt unregistered [ 1632.217300] Key type ._llcrypt unregistered [ 1663.393084] Key type ._llcrypt registered [ 1663.399567] Key type .llcrypt registered [ 1663.516392] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1664.946496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1664.999272] alg: No test for adler32 (adler32-zlib) [ 1666.101841] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1666.426810] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 1668.119322] Key type lgssc registered [ 1669.109754] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1676.969390] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1677.006749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1678.489550] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1682.698599] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1690.630714] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1697.774449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1698.189262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1703.289493] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1703.442206] Lustre: 59346:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1703.452388] Lustre: 59346:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1703.464538] Lustre: 59346:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1703.473189] Lustre: 59346:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1703.478430] Lustre: 59346:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1703.484454] Lustre: 59346:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1704.999462] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1714.049166] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1719.455611] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1724.520362] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1759.731550] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1759.744240] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1759.774277] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1763.493384] Lustre: server umount lustre-OST0000 complete [ 1768.819954] Lustre: server umount lustre-MDT0000 complete [ 1783.143343] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1786.178782] Key type lgssc unregistered [ 1786.449049] LNet: 60983:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1786.468658] LNetError: 60983:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1787.495199] LNet: Removed LNI 192.168.201.141@tcp [ 1788.771149] Key type .llcrypt unregistered [ 1788.774576] Key type ._llcrypt unregistered [ 1822.803667] Key type ._llcrypt registered [ 1822.807803] Key type .llcrypt registered [ 1822.988147] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1824.222480] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1824.278239] alg: No test for adler32 (adler32-zlib) [ 1825.413884] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1825.706317] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 1827.399861] Key type lgssc registered [ 1828.285796] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1836.376976] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1836.414833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1838.097412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1843.257424] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1852.757241] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1860.513606] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1860.808264] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1863.917074] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1863.983415] Lustre: 63834:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1863.991804] Lustre: 63834:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1863.998063] Lustre: 63834:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1864.004337] Lustre: 63834:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1864.010323] Lustre: 63834:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1864.015706] Lustre: 63834:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1866.825769] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1874.314202] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1879.580243] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1889.763581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1889.777364] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1889.792665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1893.249689] Lustre: server umount lustre-OST0000 complete [ 1896.834965] Lustre: server umount lustre-MDT0000 complete [ 1908.562360] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1911.154829] Key type lgssc unregistered [ 1911.400674] LNet: 65392:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1911.409509] LNetError: 65392:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1911.425233] LNet: Removed LNI 192.168.201.141@tcp [ 1912.410564] Key type .llcrypt unregistered [ 1912.419620] Key type ._llcrypt unregistered [ 1929.361213] Key type ._llcrypt registered [ 1929.363702] Key type .llcrypt registered [ 1929.508557] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 1944.728723] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1945.964833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1946.002178] alg: No test for adler32 (adler32-zlib) [ 1947.155954] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1947.376524] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 1949.015394] Key type lgssc registered [ 1950.120344] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1992.227679] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1992.311090] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1993.670797] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1993.719137] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1993.821158] Lustre: lustre-MDT0000: new disk, initializing [ 1993.909708] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1993.945193] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1999.358500] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2011.975400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2012.170601] Lustre: 68563:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2012.250734] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2012.255977] Lustre: Skipped 1 previous similar message [ 2012.327093] Lustre: lustre-MDT0001: new disk, initializing [ 2012.434682] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2012.508124] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2012.521030] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2017.448912] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2030.427464] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2039.478727] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2047.217678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2047.479491] Lustre: lustre-OST0000: new disk, initializing [ 2047.482288] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2047.487889] Lustre: 70538:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2047.580058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2053.113352] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2055.784175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2055.792705] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2055.905619] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2061.760827] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2066.208550] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2066.489767] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2071.107354] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2071.287569] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2076.640639] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2076.643082] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2076.645848] LustreError: Skipped 1 previous similar message [ 2076.655649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2076.664864] Lustre: Skipped 1 previous similar message [ 2081.780966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2081.787699] Lustre: Skipped 1 previous similar message [ 2086.887407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2086.893980] Lustre: Skipped 1 previous similar message [ 2087.393279] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2087.570755] Lustre: server umount lustre-OST0000 complete [ 2092.001460] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2092.002913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2092.018206] Lustre: Skipped 2 previous similar messages [ 2092.036649] Lustre: Skipped 1 previous similar message [ 2097.057433] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2097.065071] Lustre: Skipped 3 previous similar messages [ 2099.724015] Lustre: server umount lustre-MDT0000 complete [ 2103.800553] LustreError: 68556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790781700 with bad export cookie 7226793045396430805 [ 2103.818096] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2104.127590] Lustre: server umount lustre-MDT0001 complete [ 2113.687327] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 11:21:49 (1790781709) [ 2124.552648] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2162.711338] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2173.268425] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2210.359256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2210.639750] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2210.687760] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2210.764108] Lustre: lustre-MDT0000: new disk, initializing [ 2210.884891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2210.930032] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2215.404745] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2226.681863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2226.817647] Lustre: 76555:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2226.870088] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2226.874018] Lustre: Skipped 1 previous similar message [ 2226.968041] Lustre: lustre-MDT0001: new disk, initializing [ 2227.061230] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2227.089962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2227.105448] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2231.081455] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2241.495749] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2249.530370] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2256.126608] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2256.313384] Lustre: lustre-OST0000: new disk, initializing [ 2256.316164] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2256.320273] Lustre: 78523:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2256.385678] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2262.646510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2262.663560] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2262.820609] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2263.591760] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2275.922964] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2281.959845] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2282.207699] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2288.118941] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2288.342224] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2293.737986] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2293.739358] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2293.758622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2293.763523] Lustre: Skipped 1 previous similar message [ 2293.783592] Lustre: Skipped 1 previous similar message [ 2296.473507] Lustre: server umount lustre-OST0000 complete [ 2303.976616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2303.978386] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2304.003769] LustreError: Skipped 1 previous similar message [ 2313.186458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2313.194607] Lustre: Skipped 5 previous similar messages [ 2315.231325] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2315.583361] Lustre: server umount lustre-MDT0000 complete [ 2318.308198] LustreError: 77746:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2318.323622] LustreError: 77746:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2318.839074] LustreError: 76548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790781915 with bad export cookie 7226793045396431687 [ 2318.841109] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2318.848363] LustreError: 76548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2319.140996] Lustre: server umount lustre-MDT0001 complete [ 2327.229518] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 11:25:22 (1790781922) [ 2333.896531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2334.343800] LustreError: 80941:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2334.443820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2339.125576] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2339.810767] LustreError: 80942:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2344.928867] LustreError: 80941:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2347.164414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2351.927986] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2359.662677] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2364.920596] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2371.158211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2371.432098] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2371.443572] Lustre: Skipped 1 previous similar message [ 2377.275436] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2383.721114] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2387.189134] Lustre: 83612:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting 0f0d6af7-5d7c-47f3-a239-98340f3ba4e6 at adminstrative request [ 2391.011049] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2391.040716] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2401.258562] Lustre: MGS: Client 0f0d6af7-5d7c-47f3-a239-98340f3ba4e6 (at 0@lo) reconnecting [ 2401.274612] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x644ab9fc501e26b9 to 0x644ab9fc501e2959 [ 2401.291196] Lustre: MGC192.168.201.141@tcp: Connection restored to 0@lo (at 0@lo) [ 2424.114457] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2424.346178] Lustre: lustre-OST0001: new disk, initializing [ 2424.349494] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2424.353874] Lustre: 84003:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2431.165849] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2431.640096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2431.680931] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2431.808203] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2441.246557] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2447.330863] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2447.334776] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2447.345641] LustreError: Skipped 1 previous similar message [ 2447.356866] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2447.373969] Lustre: Skipped 2 previous similar messages [ 2447.390514] Lustre: Skipped 1 previous similar message [ 2451.525060] Lustre: server umount lustre-OST0001 complete [ 2452.451413] LustreError: 84388:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2452.470656] LustreError: 84388:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2457.577198] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2457.581082] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2457.591732] Lustre: Skipped 1 previous similar message [ 2462.591461] Lustre: server umount lustre-OST0000 complete [ 2472.711577] Lustre: server umount lustre-MDT0000 complete [ 2472.933451] LustreError: 81669:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2472.946909] LustreError: 81669:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2475.927944] LustreError: 80923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782073 with bad export cookie 7226793045396433241 [ 2475.952768] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2476.237630] Lustre: server umount lustre-MDT0001 complete [ 2484.857450] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 11:28:00 (1790782080) [ 2486.584548] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2488.812930] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 11:28:04 (1790782084) [ 2490.360333] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2492.352396] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 11:28:07 (1790782087) [ 2493.982441] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2495.920508] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 11:28:11 (1790782091) [ 2497.513451] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2499.435713] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 11:28:15 (1790782095) [ 2501.177251] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2506.665322] loop: module loaded [ 2506.686245] loop0: detected capacity change from 0 to 4294967296 [ 2506.740968] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2508.261779] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2511.148811] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2514.337468] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2539.018844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2539.051551] LustreError: 88125:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2539.058580] LustreError: 88125:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2539.066382] LustreError: 88125:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2539.071619] LustreError: 88125:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2557.542798] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2569.494925] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2608.165469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2608.484635] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2608.555360] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2608.640956] Lustre: lustre-MDT0000: new disk, initializing [ 2608.760644] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2608.767229] Lustre: Skipped 1 previous similar message [ 2608.778865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2614.269868] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2626.565408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2626.802909] Lustre: 91002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2627.296188] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2633.867606] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2645.070727] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2650.985862] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2657.926705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2658.188765] Lustre: 92970:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2665.198853] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2665.545353] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2665.654545] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2674.318394] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2679.011718] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2679.305971] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2684.520962] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2684.769860] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2691.554627] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2691.566110] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2691.575364] Lustre: Skipped 2 previous similar messages [ 2691.590111] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2691.597946] Lustre: Skipped 5 previous similar messages [ 2701.279163] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2701.584718] Lustre: server umount lustre-OST0000 complete [ 2712.492263] Lustre: server umount lustre-MDT0000 complete [ 2716.601429] LustreError: 90995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782313 with bad export cookie 7226793045396433857 [ 2716.605486] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2716.611639] LustreError: 90995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2717.102638] Lustre: server umount lustre-MDT0001 complete [ 2726.454588] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 11:32:01 (1790782321) [ 2733.972803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2734.266143] LustreError: 95386:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2738.831995] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2747.879676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2748.396671] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2748.409482] Lustre: Skipped 3 previous similar messages [ 2754.487167] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2761.797442] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2767.457611] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2774.366654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2780.723753] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2788.298785] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2793.574272] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2793.898804] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2798.580695] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2798.755319] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2805.363796] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2805.548144] Lustre: lustre-OST0001: new disk, initializing [ 2805.550061] Lustre: Skipped 2 previous similar messages [ 2805.554675] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2805.557538] Lustre: Skipped 3 previous similar messages [ 2805.561293] Lustre: 98756:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2812.407868] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2814.596184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2814.607108] Lustre: Skipped 2 previous similar messages [ 2814.612484] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2814.715268] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2821.942604] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2827.837606] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2828.266707] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2833.308604] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2833.587618] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2855.727325] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2886.241726] Lustre: Failing over lustre-MDT0000 [ 2886.629477] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2886.638489] Lustre: Skipped 3 previous similar messages [ 2886.644186] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2886.648732] Lustre: Skipped 6 previous similar messages [ 2886.776283] Lustre: server umount lustre-MDT0000 complete [ 2888.233558] LustreError: 95383:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2888.258274] LustreError: 95383:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2907.103899] Lustre: 66778:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790782488/real 1790782488] req@00000000814a3d37 x1877770548560128/t0(0) o400->MGC192.168.201.141@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790782504 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2907.141684] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2912.843443] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2912.846478] LDISKFS-fs (dm-0): recovery complete [ 2912.856107] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2917.357506] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x644ab9fc501ef3d4 [ 2917.369033] LustreError: 101591:0:(ldlm_resource.c:1227:ldlm_resource_complain()) MGC192.168.201.141@tcp: namespace resource [0x65727473756c:0x5:0x0].0x0 (00000000baf3e490) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 2917.393983] Lustre: MGC192.168.201.141@tcp: Connection restored to 0@lo (at 0@lo) [ 2917.946548] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2919.940770] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2919.941593] Lustre: 95383:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2922.989821] Lustre: 96114:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2923.003088] Lustre: 96114:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2923.020778] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2923.155187] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2923.382824] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2923.488674] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2923.831567] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2923.843641] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2924.160272] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2924.178056] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2924.188459] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2924.206972] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2924.859677] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2924.879522] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2925.208177] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2925.215872] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2926.203832] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2926.219081] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 5 previous similar messages [ 2926.880043] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2926.895933] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 2927.216395] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2927.224888] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 2930.417565] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2930.439037] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2931.090361] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2931.109899] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2931.431431] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2931.444872] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2938.725861] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2938.731857] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 25 previous similar messages [ 2939.370770] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2939.386773] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 2939.704802] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2939.712557] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 2954.874971] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2954.885129] Lustre: 101615:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 50 previous similar messages [ 2955.542630] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2955.559984] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2955.879074] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2955.889414] LustreError: 101615:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2979.079166] Lustre: 101615:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2979.086825] Lustre: 101615:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2979.091134] Lustre: 101615:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 0455dd0f-ce8b-4a04-8ad4-b3bb03780e64@192.168.201.41@tcp [ 2979.097961] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2979.102677] Lustre: 101615:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@000000001f2255c7 x1877770532622720/t0(8589934772) o35->0455dd0f-ce8b-4a04-8ad4-b3bb03780e64@192.168.201.41@tcp:351/0 lens 392/0 e 0 to 0 dl 1790782586 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2979.116574] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2979.127833] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2979.136429] Lustre: lustre-MDT0000: Denying connection for new client 0455dd0f-ce8b-4a04-8ad4-b3bb03780e64 (at 192.168.201.41@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2979.147491] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2979.150995] Lustre: Skipped 2 previous similar messages [ 2979.155831] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2979.180815] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2979.180825] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2999.266496] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2999.270801] LustreError: Skipped 1 previous similar message [ 2999.272523] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2999.280426] Lustre: Skipped 3 previous similar messages [ 3002.779171] Lustre: server umount lustre-OST0000 complete [ 3004.904492] LustreError: 97209:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3004.942036] LustreError: 97209:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 37 previous similar messages [ 3012.845983] Lustre: server umount lustre-OST0001 complete [ 3022.990425] Lustre: server umount lustre-MDT0000 complete [ 3027.074197] LustreError: 95368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782624 with bad export cookie 7226793045396485076 [ 3027.076705] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3027.086025] LustreError: 95368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3027.476840] Lustre: server umount lustre-MDT0001 complete [ 3036.459064] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 11:37:12 (1790782632) [ 3043.403494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3043.410821] Lustre: *** cfs_fail_loc=197, val=0*** [ 3058.975707] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3064.096491] LustreError: 103589:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.141@tcp: failed processing log, type 4: rc = -110 [ 3069.230348] Lustre: *** cfs_fail_loc=197, val=0*** [ 3069.234924] Lustre: Skipped 65 previous similar messages [ 3089.826529] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3089.832453] Lustre: Skipped 3 previous similar messages [ 3096.801185] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3104.492760] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3107.793322] Lustre: server umount lustre-OST0000 complete [ 3120.192682] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 11:38:35 (1790782715) [ 3142.205141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3142.731204] LustreError: 105206:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3142.766289] LustreError: 105206:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 3148.188424] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3160.731589] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3165.354507] LustreError: 105237:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3165.362250] LustreError: 105237:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3165.367853] LustreError: 105237:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 3165.861973] Lustre: server umount lustre-MDT0000 complete [ 3201.840398] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3212.919412] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3249.354656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3249.578399] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3249.604799] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3249.664697] Lustre: lustre-MDT0000: new disk, initializing [ 3249.709770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3254.731372] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3266.434609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3266.555665] Lustre: 109414:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3266.775965] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3271.870513] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3282.195746] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3288.694310] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3296.605842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3296.927327] Lustre: 111384:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3305.064308] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3306.365929] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3316.211693] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3321.779409] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3322.077808] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3327.058773] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3327.302937] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3332.077534] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3332.086098] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3332.111791] LustreError: Skipped 3 previous similar messages [ 3332.115624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3332.144949] Lustre: Skipped 6 previous similar messages [ 3332.161607] Lustre: Skipped 9 previous similar messages [ 3335.827425] Lustre: server umount lustre-OST0000 complete [ 3354.079248] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3354.353669] Lustre: server umount lustre-MDT0000 complete [ 3358.144603] LustreError: 109407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782955 with bad export cookie 7226793045396495695 [ 3358.157123] LustreError: 109407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3358.161939] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3358.530695] Lustre: server umount lustre-MDT0001 complete [ 3369.423177] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 11:42:44 (1790782964) [ 3379.632831] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 3382.487573] Key type lgssc unregistered [ 3382.734526] LNet: 113871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3382.740557] LNetError: 113871:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3383.787887] LNet: Removed LNI 192.168.201.141@tcp [ 3384.662522] Key type .llcrypt unregistered [ 3384.666514] Key type ._llcrypt unregistered [ 3404.987057] Key type ._llcrypt registered [ 3404.990735] Key type .llcrypt registered [ 3405.159304] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3425.500936] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3426.553945] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3426.579799] alg: No test for adler32 (adler32-zlib) [ 3427.806892] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 3428.381587] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 3430.135966] Key type lgssc registered [ 3431.358739] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3476.073581] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3476.143775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3477.495513] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3477.567152] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3477.641969] Lustre: lustre-MDT0000: new disk, initializing [ 3477.705659] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3477.734238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3482.875445] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3493.857681] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3494.089756] Lustre: lustre-OST0000: new disk, initializing [ 3494.092607] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3494.095127] Lustre: Skipped 1 previous similar message [ 3494.099319] Lustre: 118063:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3494.160696] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3494.758699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3494.785702] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3494.870419] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3501.494828] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3512.136670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3512.285532] Lustre: lustre-OST0001: new disk, initializing [ 3512.289732] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3512.299551] Lustre: 118928:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3512.398632] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3517.592322] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3517.607253] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3517.657340] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3519.605388] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3533.280758] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3533.291997] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3535.614321] Lustre: server umount lustre-MDT0000 complete [ 3542.653871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3547.823247] LustreError: 117250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783144 with bad export cookie 4470578484762186449 [ 3547.828119] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3547.841209] LustreError: 117250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3547.939893] Lustre: server umount lustre-OST0000 complete [ 3552.452770] Lustre: server umount lustre-OST0001 complete [ 3561.484738] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 11:45:56 (1790783156) [ 3584.828845] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 11:46:20 (1790783180) [ 3595.167378] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3617.298107] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 11:46:52 (1790783212) [ 3666.820899] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3679.308851] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3724.161943] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3724.554779] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3724.588791] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3724.688971] Lustre: lustre-MDT0000: new disk, initializing [ 3724.801685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3724.823718] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3731.029217] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3741.812179] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3741.938906] Lustre: 126682:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3741.989624] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3741.993746] Lustre: Skipped 1 previous similar message [ 3742.053340] Lustre: lustre-MDT0001: new disk, initializing [ 3742.142236] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3742.199237] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3742.221815] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3748.776896] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3760.606868] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3767.337764] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3774.928656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3775.471664] Lustre: lustre-OST0000: new disk, initializing [ 3775.476829] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3775.489623] Lustre: 128653:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3775.621441] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3783.835229] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3784.831615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3784.848070] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3785.050433] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3794.986791] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3800.210576] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3800.503547] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3807.078620] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3807.483147] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3810.787425] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3810.794282] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3810.802386] LustreError: Skipped 1 previous similar message [ 3810.809739] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3810.809747] Lustre: Skipped 1 previous similar message [ 3810.859425] Lustre: Skipped 2 previous similar messages [ 3816.264512] Lustre: server umount lustre-OST0000 complete [ 3821.026737] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3821.045531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3821.071743] Lustre: Skipped 2 previous similar messages [ 3826.144476] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3826.160177] Lustre: Skipped 1 previous similar message [ 3827.211565] Lustre: server umount lustre-MDT0000 complete [ 3831.042445] LustreError: 126676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783428 with bad export cookie 4470578484762187674 [ 3831.052597] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3831.052728] LustreError: 126676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3831.367934] Lustre: server umount lustre-MDT0001 complete [ 3840.795059] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:50:36 (1790783436) [ 3847.371764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3848.047854] LustreError: 131069:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3848.071971] LustreError: 131069:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3848.175580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3853.283208] LustreError: 131070:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3853.557635] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3858.404327] LustreError: 131069:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3862.062279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3867.900869] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3876.941414] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3884.350318] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3891.667780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3891.913770] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3891.931228] Lustre: Skipped 1 previous similar message [ 3898.396935] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3906.861673] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3911.189226] Lustre: *** cfs_fail_loc=159, val=0*** [ 3927.626233] Lustre: lustre-MDT0000: Client f203962c-62da-4561-a336-23466706b155 (at 192.168.201.41@tcp) reconnecting [ 3929.166060] Lustre: *** cfs_fail_loc=159, val=0*** [ 3929.167847] Lustre: Skipped 5 previous similar messages [ 3945.569446] Lustre: lustre-MDT0000: Client f203962c-62da-4561-a336-23466706b155 (at 192.168.201.41@tcp) reconnecting [ 3963.878616] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3963.886162] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3963.892387] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3963.900073] Lustre: Skipped 2 previous similar messages [ 3968.758078] Lustre: server umount lustre-OST0000 complete [ 3974.112523] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3974.121109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3974.121117] Lustre: Skipped 1 previous similar message [ 3974.129709] Lustre: Skipped 1 previous similar message [ 3986.399375] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3986.500391] Lustre: server umount lustre-MDT0000 complete [ 3989.475205] LustreError: 133602:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3989.502121] LustreError: 133602:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3990.308952] LustreError: 131050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783587 with bad export cookie 4470578484762188556 [ 3990.316568] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3990.321879] LustreError: 131050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3990.728921] Lustre: server umount lustre-MDT0001 complete [ 4003.232794] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4006.307590] Key type lgssc unregistered [ 4006.687748] LNet: 134845:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4006.702304] LNetError: 134845:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4006.740965] LNet: Removed LNI 192.168.201.141@tcp [ 4007.701834] Key type .llcrypt unregistered [ 4007.703221] Key type ._llcrypt unregistered [ 4035.187647] Key type ._llcrypt registered [ 4035.189486] Key type .llcrypt registered [ 4035.287597] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4036.248662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4036.270422] alg: No test for adler32 (adler32-zlib) [ 4037.346725] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4037.568691] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4039.239332] Key type lgssc registered [ 4040.635566] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4048.620837] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4048.664292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4050.126392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4054.569348] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4060.652682] LustreError: 136278:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4060.672827] LustreError: 136278:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4063.954652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4064.656948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4070.484676] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4078.727213] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4085.234548] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4092.743523] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4093.093755] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4095.188264] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4099.616292] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4107.816656] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4116.972173] Lustre: *** cfs_fail_loc=159, val=0*** [ 4133.416160] Lustre: lustre-MDT0000: Client 3c2a039f-4528-4be1-a40e-f04a767d9d80 (at 192.168.201.41@tcp) reconnecting [ 4136.151943] Lustre: *** cfs_fail_loc=159, val=0*** [ 4152.365580] Lustre: lustre-MDT0001: Client 3c2a039f-4528-4be1-a40e-f04a767d9d80 (at 192.168.201.41@tcp) reconnecting [ 4154.809496] Lustre: *** cfs_fail_loc=159, val=0*** [ 4154.811219] Lustre: Skipped 3 previous similar messages [ 4170.312831] Lustre: lustre-MDT0001: Client 3c2a039f-4528-4be1-a40e-f04a767d9d80 (at 192.168.201.41@tcp) reconnecting [ 4173.769377] Lustre: 139318:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4176.556933] Lustre: *** cfs_fail_loc=159, val=0*** [ 4176.565727] Lustre: Skipped 4 previous similar messages [ 4192.296512] Lustre: lustre-MDT0000: Client c8e2d178-955e-4b0c-9838-9c1107d6ac93 (at 192.168.201.41@tcp) reconnecting [ 4193.859104] Lustre: *** cfs_fail_loc=159, val=0*** [ 4193.863934] Lustre: Skipped 13 previous similar messages [ 4209.219118] Lustre: lustre-MDT0000: Client c8e2d178-955e-4b0c-9838-9c1107d6ac93 (at 192.168.201.41@tcp) reconnecting [ 4210.871468] Lustre: 139612:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4216.809788] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4216.810101] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4216.820376] LustreError: Skipped 1 previous similar message [ 4216.838751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4218.337334] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4218.341251] LustreError: Skipped 1 previous similar message [ 4218.344189] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4218.354552] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4220.391692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4220.395483] Lustre: Skipped 1 previous similar message [ 4222.171782] Lustre: server umount lustre-OST0000 complete [ 4228.578440] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4228.586723] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4228.605969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4230.630550] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4232.449081] Lustre: server umount lustre-MDT0000 complete [ 4235.746394] LustreError: 138965:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4235.765805] LustreError: 138965:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4236.090852] LustreError: 136257:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790783833 with bad export cookie 7504091758637530448 [ 4236.102023] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4236.106482] LustreError: 136257:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4236.402546] Lustre: server umount lustre-MDT0001 complete [ 4248.458105] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4251.530335] Key type lgssc unregistered [ 4251.850867] LNet: 140663:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4251.858146] LNetError: 140663:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4251.879874] LNet: Removed LNI 192.168.201.141@tcp [ 4252.574686] Key type .llcrypt unregistered [ 4252.585843] Key type ._llcrypt unregistered [ 4277.775193] Key type ._llcrypt registered [ 4277.777593] Key type .llcrypt registered [ 4277.854562] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4278.958594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4278.974604] alg: No test for adler32 (adler32-zlib) [ 4279.989940] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4280.269337] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4282.039284] Key type lgssc registered [ 4283.153668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4291.879760] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4291.921575] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4293.598412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4298.777415] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4303.847740] LustreError: 142094:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4303.868893] LustreError: 142094:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4308.185631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4308.736876] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4314.821935] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4325.029570] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4332.413962] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4339.893143] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4340.195070] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4346.615511] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4353.532122] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4353.708744] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4363.790579] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4371.074852] Lustre: 144703:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4374.128725] Lustre: 144751:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4375.761606] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4376.657958] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4380.482685] Lustre: 144897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4381.430060] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4382.218865] Lustre: 144995:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4389.344416] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4389.350954] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4389.356852] LustreError: Skipped 1 previous similar message [ 4389.359166] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4389.375686] Lustre: Skipped 1 previous similar message [ 4392.829913] Lustre: server umount lustre-OST0000 complete [ 4399.588974] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4399.596432] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4399.605211] Lustre: Skipped 1 previous similar message [ 4399.618595] Lustre: Skipped 2 previous similar messages [ 4402.516568] Lustre: server umount lustre-MDT0000 complete [ 4404.705071] LustreError: 142090:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4404.720770] LustreError: 142090:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4406.071157] LustreError: 143936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790784003 with bad export cookie 16419350655775450416 [ 4406.072632] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4406.084017] LustreError: 143936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4406.423440] Lustre: server umount lustre-MDT0001 complete [ 4417.341145] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4420.431827] Key type lgssc unregistered [ 4420.849664] LNet: 146055:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4420.856201] LNetError: 146055:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4420.897842] LNet: Removed LNI 192.168.201.141@tcp [ 4422.138499] Key type .llcrypt unregistered [ 4422.141540] Key type ._llcrypt unregistered [ 4451.936620] Key type ._llcrypt registered [ 4451.939199] Key type .llcrypt registered [ 4452.045984] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4453.829649] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4453.882405] alg: No test for adler32 (adler32-zlib) [ 4454.949785] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4455.265666] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4456.961182] Key type lgssc registered [ 4458.233183] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4466.368624] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4466.396648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4468.159968] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4473.829455] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4478.436082] LustreError: 147496:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4478.491712] LustreError: 147496:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4483.573964] LustreError: 147497:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4485.527328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4486.164849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4491.875034] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4500.283872] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4507.791904] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4515.325363] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4515.674297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4523.258919] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4531.073981] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4531.979562] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4538.226369] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4543.310042] Lustre: *** cfs_fail_loc=159, val=0*** [ 4559.943342] Lustre: lustre-MDT0000: Client b3284b4a-f7b0-4c69-9962-fa44bd92ee45 (at 192.168.201.41@tcp) reconnecting [ 4579.363658] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4579.367221] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4579.370937] Lustre: Skipped 1 previous similar message [ 4579.378354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4579.396344] Lustre: Skipped 1 previous similar message [ 4580.094470] Lustre: server umount lustre-OST0000 complete [ 4584.419236] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4584.439474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4584.454231] Lustre: Skipped 1 previous similar message [ 4589.538064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4589.542868] Lustre: Skipped 1 previous similar message [ 4590.788717] Lustre: server umount lustre-MDT0000 complete [ 4594.659759] LustreError: 147496:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4594.671084] LustreError: 147496:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4595.351300] LustreError: 147477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790784192 with bad export cookie 9826121711922945818 [ 4595.375886] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4595.955914] Lustre: server umount lustre-MDT0001 complete [ 4609.671516] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4613.315312] Key type lgssc unregistered [ 4613.590034] LNet: 151176:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4613.606117] LNetError: 151176:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4613.626493] LNet: Removed LNI 192.168.201.141@tcp [ 4614.492252] Key type .llcrypt unregistered [ 4614.495033] Key type ._llcrypt unregistered [ 4648.017604] Key type ._llcrypt registered [ 4648.022889] Key type .llcrypt registered [ 4648.161670] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4649.592526] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4649.722746] alg: No test for adler32 (adler32-zlib) [ 4650.730411] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4650.970561] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4652.671289] Key type lgssc registered [ 4654.396596] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4663.643357] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4663.681322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4665.170783] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4670.721910] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4675.554030] LustreError: 152606:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4675.577188] LustreError: 152606:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4680.689321] LustreError: 152607:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4682.319022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4683.070185] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4690.614621] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4703.240609] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4711.452832] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4719.296167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4719.496995] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4726.749346] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4727.689616] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4736.911698] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4736.990542] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4757.866413] Lustre: lustre-OST0000: evicting dbcf22c8-48bf-4a62-8131-3c95af67ad16 (at 192.168.201.41@tcp) by administrative request [ 4757.879047] Lustre: Skipped 1 previous similar message [ 4788.200577] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4788.213359] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4788.213814] Lustre: Skipped 1 previous similar message [ 4788.215980] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4788.241661] Lustre: Skipped 1 previous similar message [ 4793.271898] Lustre: server umount lustre-OST0000 complete [ 4798.432448] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4798.457278] Lustre: Skipped 1 previous similar message [ 4798.457880] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4798.482777] Lustre: Skipped 1 previous similar message [ 4803.553387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4803.562227] Lustre: Skipped 1 previous similar message [ 4803.974199] Lustre: server umount lustre-MDT0000 complete [ 4808.673420] LustreError: 152607:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4808.680439] LustreError: 152607:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4809.332788] LustreError: 154461:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790784406 with bad export cookie 13188221815536590678 [ 4809.338956] LustreError: 154461:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4809.341427] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4809.819341] Lustre: server umount lustre-MDT0001 complete [ 4823.329289] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4826.395419] Key type lgssc unregistered [ 4826.899564] LNet: 156680:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4826.921517] LNetError: 156680:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4827.946307] LNet: Removed LNI 192.168.201.141@tcp [ 4829.525132] Key type .llcrypt unregistered [ 4829.526929] Key type ._llcrypt unregistered [ 4924.142211] Key type ._llcrypt registered [ 4924.151710] Key type .llcrypt registered [ 4924.249699] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4925.906153] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4925.967642] alg: No test for adler32 (adler32-zlib) [ 4927.158729] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4927.416314] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4929.135214] Key type lgssc registered [ 4930.566956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4939.626753] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4939.638718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4941.241902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4946.062762] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4951.532346] LustreError: 159190:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4951.550258] LustreError: 159190:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4955.466817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4956.053234] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4961.061165] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4970.307580] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4977.176464] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4984.279284] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4984.893251] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4985.986938] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4992.434800] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5001.357602] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5003.742107] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5017.064734] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5017.067331] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5017.079292] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5017.081557] LustreError: Skipped 1 previous similar message [ 5017.095233] Lustre: Skipped 1 previous similar message [ 5017.374829] Lustre: server umount lustre-OST0000 complete [ 5022.179780] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5022.200183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5022.209692] Lustre: Skipped 2 previous similar messages [ 5022.229256] Lustre: Skipped 1 previous similar message [ 5027.299985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5027.320900] Lustre: Skipped 1 previous similar message [ 5027.912543] Lustre: server umount lustre-MDT0000 complete [ 5032.105871] LustreError: 159170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790784629 with bad export cookie 17971418966694749218 [ 5032.118724] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5032.463981] Lustre: server umount lustre-MDT0001 complete [ 5046.658173] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 5050.072942] Key type lgssc unregistered [ 5050.435076] LNet: 162734:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5050.439660] LNetError: 162734:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5051.500099] LNet: Removed LNI 192.168.201.141@tcp [ 5052.501147] Key type .llcrypt unregistered [ 5052.503393] Key type ._llcrypt unregistered [ 5079.983675] Key type ._llcrypt registered [ 5079.986245] Key type .llcrypt registered [ 5080.165863] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5081.456586] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5081.666405] alg: No test for adler32 (adler32-zlib) [ 5083.295890] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5083.640259] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5085.407384] Key type lgssc registered [ 5087.841148] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5113.135692] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 12:11:48 (1790784708) [ 5131.598671] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 5143.408801] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5183.819425] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5183.858961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5185.170582] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5185.237513] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5185.336923] Lustre: lustre-MDT0000: new disk, initializing [ 5185.478903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5185.505463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5190.717037] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5201.925545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5202.106564] Lustre: 167015:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5202.198059] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5202.202914] Lustre: Skipped 1 previous similar message [ 5202.303086] Lustre: lustre-MDT0001: new disk, initializing [ 5202.424168] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5202.470791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5202.486361] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5207.818855] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5218.308060] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5225.358463] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5232.810444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5233.189092] Lustre: lustre-OST0000: new disk, initializing [ 5233.195955] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5233.209652] Lustre: 168985:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5233.396993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5240.982540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5240.991169] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5241.159877] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5243.438209] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5254.317319] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5266.924360] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5266.927376] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5266.935744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5266.969928] Lustre: Skipped 1 previous similar message [ 5269.910515] Lustre: server umount lustre-OST0000 complete [ 5274.081240] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5274.091524] LustreError: Skipped 1 previous similar message [ 5274.098134] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5274.117499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5274.125197] Lustre: Skipped 1 previous similar message [ 5277.172055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5277.176872] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5280.409289] Lustre: server umount lustre-MDT0000 complete [ 5284.936959] LustreError: 167008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790784882 with bad export cookie 15984523267616729048 [ 5284.957448] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5284.957554] LustreError: 167008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5285.385874] Lustre: server umount lustre-MDT0001 complete [ 5297.652262] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 5300.394583] Key type lgssc unregistered [ 5300.618873] LNet: 170888:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5300.624443] LNetError: 170888:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5300.635047] LNet: Removed LNI 192.168.201.141@tcp [ 5301.662876] Key type .llcrypt unregistered [ 5301.665192] Key type ._llcrypt unregistered [ 5331.855730] Key type ._llcrypt registered [ 5331.857929] Key type .llcrypt registered [ 5331.961949] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5333.006551] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5333.079146] alg: No test for adler32 (adler32-zlib) [ 5334.410928] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5334.763613] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5336.479228] Key type lgssc registered [ 5337.754257] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5345.958629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5345.995631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5347.590765] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5352.438840] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5357.027148] LustreError: 172320:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5357.039162] LustreError: 172320:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5361.450259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5361.783204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5366.974154] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5375.806595] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5382.745859] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5390.337992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5390.858813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5392.936079] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5397.584493] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5406.157045] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5415.092267] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5415.147463] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5415.185947] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5415.199166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.248523] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5416.833297] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5416.862975] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5416.883845] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5416.899913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.949605] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5418.207379] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5418.240764] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5418.263691] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5418.269540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.280894] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5419.574559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.856611] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5420.862142] Lustre: Skipped 1 previous similar message [ 5420.878225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5420.886296] Lustre: Skipped 1 previous similar message [ 5420.889665] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5420.894249] Lustre: Skipped 1 previous similar message [ 5420.897460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.906151] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5420.914084] Lustre: Skipped 1 previous similar message [ 5422.125259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.583025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.799071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.032730] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5426.044557] Lustre: Skipped 3 previous similar messages [ 5426.069630] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5426.088350] Lustre: Skipped 3 previous similar messages [ 5426.096310] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5426.099379] Lustre: Skipped 3 previous similar messages [ 5426.111387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.130844] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5426.142773] Lustre: Skipped 3 previous similar messages [ 5427.248830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.042559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.150273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.120500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.970202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.145616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.299637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.053914] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5434.058131] Lustre: Skipped 7 previous similar messages [ 5434.072183] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5434.086089] Lustre: Skipped 7 previous similar messages [ 5434.090966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.044076] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5435.050389] Lustre: Skipped 8 previous similar messages [ 5435.053851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.061141] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5435.067631] Lustre: Skipped 8 previous similar messages [ 5435.838314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.846606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.895406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.759405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.605507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.480898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.464474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.292604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.157346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.939353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.655688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.581587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.370975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.254253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.137837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.136587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.050868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.159346] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5451.163523] Lustre: Skipped 18 previous similar messages [ 5451.179873] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5451.191161] Lustre: Skipped 18 previous similar messages [ 5451.196573] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5451.198626] Lustre: Skipped 17 previous similar messages [ 5451.202979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.209619] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5451.220195] Lustre: Skipped 17 previous similar messages [ 5452.321669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.383063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.376480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.605120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.877266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.014195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.258882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.215386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.067886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.002966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.978251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.722751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.504943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.240577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.218629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.915439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.038156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.955455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.972815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.086672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.044117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.825296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.697852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.577474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.548735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.398615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.208975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.188458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.055106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.990336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.835654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.983598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.968429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.984424] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5483.991864] Lustre: Skipped 33 previous similar messages [ 5484.005412] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5484.015511] Lustre: Skipped 33 previous similar messages [ 5484.019469] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5484.027523] Lustre: Skipped 33 previous similar messages [ 5484.033126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.057101] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5484.064709] Lustre: Skipped 33 previous similar messages [ 5485.018034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.035142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.849374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.879860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.932694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.927224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.658777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.543291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.684195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.461249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.561837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.450845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.299155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.135768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.074286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.952397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.037466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.068233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.143328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.232625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.523728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.568718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.597559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.588798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.486073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.633228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.614292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.563985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.656243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.757572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.725212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.633308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.532619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.417826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.529069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.612524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.658643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.609649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.518752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.432329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.480831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.391593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.464455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.431650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.380262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.021789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.063234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.996351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.865650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.634347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.692041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.657509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.564878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.421826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.308550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.199398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.236419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.195420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.104527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.124855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.069704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.945459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.020454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.041747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.059995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.114952] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5548.127808] Lustre: Skipped 65 previous similar messages [ 5548.165921] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5548.179481] Lustre: Skipped 65 previous similar messages [ 5548.199471] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5548.210190] Lustre: Skipped 65 previous similar messages [ 5548.220334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.250555] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5548.271532] Lustre: Skipped 65 previous similar messages [ 5549.131731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.401579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.620460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.507832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.594197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.479164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.508940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.499225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.463221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.471720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.549253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.513512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.348364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.257808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.222575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.300507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.223309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.103062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.907194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.893040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.666814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.615897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.519638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.495442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.368779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.279606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.275969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.314810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.018874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.035737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.936553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.920576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.825564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.783379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.729422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.747856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.476769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.445960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.427917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.434714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.291916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.394199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.334210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.365272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.249862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.104037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.090509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.035545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.080222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.091255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.983417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.921837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.923974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.701347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.650669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.479459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.319103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.035118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.842225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.651080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.539682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.493687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.434171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.112554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.313689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.060252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.857898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.640985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.482646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.309634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.283358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.089487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.936593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.793227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.923455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.858851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.778394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.636044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.678402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.518971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.508378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.426944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.633810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.868885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.015711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.333435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.393048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.439388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.201778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.141185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.346803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.317555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.281546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.246454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.313178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.416491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.367410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.705129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.798489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.030297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.139547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.623309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.978886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.380678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.742328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.485774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.788786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.133608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.440064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.682344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.861845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.809455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.102515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.080664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.376175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.277649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.400128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.202200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.606060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.121115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.182257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.957903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.788133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.496681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.698437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.652041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.797646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.876456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.893458] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5676.911249] Lustre: Skipped 128 previous similar messages [ 5676.942213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5676.949017] Lustre: Skipped 128 previous similar messages [ 5676.952477] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5676.956675] Lustre: Skipped 128 previous similar messages [ 5676.959824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.966859] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5676.972662] Lustre: Skipped 128 previous similar messages [ 5677.709150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.708173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.363173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.504842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.557192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.788620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.764323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.713084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.027497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.197345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.101994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.489451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.444524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.787544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.192463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.485608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.781739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.944722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.024394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.251794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.395539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.524803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.429682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.239554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.970794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.898453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.672344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.537613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.329619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.174396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.113965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.258756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.505679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.872592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.136332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.393807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.527491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.503122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.355968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.234162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.190563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.001893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.081242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.066540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.106221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.124369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.231226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.475337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.624987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.461320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.475682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.160577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.282900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.268388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.326536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.373592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.177276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.999338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.881920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.809365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.837995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.606942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.489472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.425334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.356565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.214954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.154333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.210666] LustreError: 172339:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5746.219609] LustreError: 172339:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5746.237976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.194458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.170672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.034971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.187810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.189124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.159293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.047913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.950830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.902278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.952634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.864750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.053347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.100355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.172324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.134730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.209264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.168549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.108996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.219330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.006207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.974135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.957216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.905681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.719907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.774056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.609248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.527495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.573742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.717859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.937374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.935681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.942669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.862283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.904875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.731987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.577547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.599143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.596058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.544543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.552035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.590614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.675722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.591902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.476802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.470667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.696358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.831859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.717783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.815688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.703143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.602382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.539101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.394931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.237097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.276552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.255640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.055832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.790943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.562036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.483500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.274503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.172727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.127824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.887549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.578338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.429487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.479271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.172401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.086817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.908562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.666184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.571578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.492770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.432515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.304079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.133222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.181285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.996627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.959736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.751844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.706749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.484622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.573427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.512866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.512507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.569063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.563324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.630698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.481100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.452063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.541513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.336258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.307493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.104632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.049835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.988756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.946616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.820283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.916396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.685762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.681868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.610791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.646343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.576657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.717619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.961593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.997853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.203373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.217471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.359759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.282244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.419247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.580254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.761105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.077910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.064781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.119555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.120624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.370216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.605686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.604608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.723436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.588784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.494986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.431558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.728856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.613298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.936785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.973834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.315089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.423334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.374719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.473989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.406770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.330923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.327282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.359225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.384659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.104345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.974605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.004411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.832351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.757667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.732391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.633678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.752421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.880604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.258050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.410632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.762833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.966074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.908080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.206464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.220985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.234383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.167532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.183449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.229127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.405297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.216579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.675803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.894519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.886030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.182657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.409108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.344711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.305722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.219890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.216834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.187216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.045805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.011505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.809338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.868227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.842296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.811361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.680723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.107583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.163435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.106385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.209926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.419886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.372044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.294828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.275358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.998442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.114585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.011681] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5933.016558] Lustre: Skipped 255 previous similar messages [ 5933.032331] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5933.039272] Lustre: Skipped 255 previous similar messages [ 5933.043705] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5933.048602] Lustre: Skipped 255 previous similar messages [ 5933.051661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.071884] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5933.080868] Lustre: Skipped 255 previous similar messages [ 5934.153521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.033511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.906564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.637484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.563610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.468865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.580403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.530198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.521468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.600791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.497302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.600888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.577559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.533303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.732582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.711642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.695095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.639250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.768346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.682080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.810935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.759748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.039327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.380203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.290199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.602747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.638166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.597988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.670645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.694759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.714823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.789689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.987110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.911583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.725850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.862032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.960916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.034657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.957562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.103644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.190839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.115515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.971638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.858923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.620861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.374586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.107703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.917615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.653774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.631199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.415548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.141864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.949358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.190391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.221268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.312955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.340786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.290831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.226895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.215413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.206186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.093837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.171755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.355467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.534381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.482526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.351201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.327344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.214529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.276604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.239425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.170432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.125051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.336464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.383315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.558402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.261074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.582562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.715812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.909637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.319817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.032158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.345721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.542492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.780144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.183865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.248278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.719209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.051193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.105139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.967608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.678541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.725943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.759583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.763777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.716027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.898651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.697481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.635135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.554436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.745602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.527190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.625085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.685585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.770872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.774465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.714721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.631336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.690550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.637395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.748726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.517889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.394973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.077493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.981184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.053970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.401092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.573632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.513588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.687492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.467187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.536635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.211397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.218809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.526799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.862320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.873275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.903347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.795174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.724254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.897964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.111328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.336875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.443147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.167598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.037409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.733439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.117876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.571126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.794909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.934856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.828272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.919325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.955112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.004289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.826249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.970187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.039408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.579383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.907529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.370749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.424706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.553060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.959642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.125739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.107571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.845676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.681353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.557523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.252285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.238931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.969966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.718570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.502980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.388590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.052692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.898375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.562426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.535408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.420642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.345142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.115044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.114729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.994649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.925675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.744825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.582870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.422437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.658825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.829674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.825296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.966832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.853938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.848103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.924653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.080442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.139376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.129399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.352455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.850820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.844885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.500132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.676498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.797319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.962375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.952981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.355946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.486399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.693887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.764415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.741742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.990954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.382276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.831943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.241805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.452417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.378996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.830948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.532032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.676543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.511700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.425865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.283307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.281202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.166271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.134895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.004815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.092427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.077796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.389890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.342628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.277813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.048972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.004867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.833585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.845915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.837632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.738320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.052406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.123136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.026303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.817211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.746745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.874314] LustreError: 172339:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6175.884428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.891756] LustreError: 172339:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6177.042245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.647436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.767495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.647628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.573856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.241275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.107641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.968930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.904412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.016177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.183925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.524496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.824755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.973330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.247877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.687892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.916749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.866535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.968883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.840711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.004936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.955836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.839831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.930268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.905394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.885044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.727103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.824257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.869042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.801795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.812485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.696446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.592905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.788242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.644603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.823633] LustreError: 172339:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6213.836109] LustreError: 172339:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6213.851952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.721438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.850032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.032435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.878824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.985386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.992864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.007706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.789946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.580582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.636380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.413532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.039240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.950405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.948432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.069062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.678132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.569552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.380806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.087811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.842410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.982465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.703618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.868113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.625949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.570772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.176610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.117953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.161492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.899849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.218204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.154858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.260393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.121269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.501064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.534215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.449191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.698245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.858692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.005802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.962326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.721073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.969066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.118294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.031829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.930899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.658495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.128697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.423839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.095954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.323096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.203822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.060523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.119196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.518066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.459474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.244847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.425911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.402174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.586893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.471585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.196473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.946042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.906227] LustreError: 172339:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6276.913163] LustreError: 172339:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6276.938418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.023934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.837919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.969745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.879339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.960578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.081993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.402661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.331453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.418140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.510667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.564505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.670863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.612789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.409800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.298107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.186689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.376230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.706905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.876186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.136289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.142477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.210342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.263071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.316082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.342065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.196725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.185479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.198289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.333289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.224872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.236631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.221418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.268255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.111931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.181086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.489961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.461764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.525878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.203127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.191052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.083555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.995131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.725383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.521599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.450974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.359069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.376893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.429833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.685423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.836556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.546669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.659438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.527033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.483895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.411060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.308507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.312191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.057991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.930466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.633313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.514651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.350271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.274494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.142147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.023717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.878878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.025611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.043272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.992223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.939575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.819608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.760666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.735438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.840976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.894640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.905848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.882764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.892110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.042601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.411816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.410909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.360605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.181144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.073645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.000686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.883231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.978866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.997481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.946514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.907024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.754934] LustreError: 172339:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6366.772625] LustreError: 172339:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6366.786413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.771123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.674181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.680039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.695866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.649356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.627237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.822727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.202524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.700484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.455267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.439907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.434528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.661663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.489354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.372054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.384681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.519756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.605058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.906993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.992301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.405757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.168768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.151307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.626055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.809679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.869241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.835748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.799441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.925551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.255026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.683681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.696088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.877028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.041175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.093431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.269915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.299648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.356394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.493701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.663867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.582525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.477628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.539053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.101547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.467318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.546371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.895053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.226767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.541553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.649330] LustreError: 172339:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6422.663458] LustreError: 172339:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6422.684322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.750200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.250245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.270450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.093387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.107705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.113754] LustreError: 174143:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6838.122584] LustreError: 172339:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7406.061112] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7406.075090] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7406.101922] Lustre: Skipped 479 previous similar messages [ 7406.109048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7407.079748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7407.088990] Lustre: Skipped 1 previous similar message [ 7410.554285] Lustre: server umount lustre-OST0000 complete [ 7414.754336] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7414.774609] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7417.312832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7420.023325] Lustre: server umount lustre-MDT0000 complete [ 7422.437469] LustreError: 172320:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7422.458183] LustreError: 172320:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7423.597244] LustreError: 172300:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790787020 with bad export cookie 10491934449364088848 [ 7423.603356] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7423.606907] LustreError: 172300:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7423.834753] Lustre: server umount lustre-MDT0001 complete [ 7436.193515] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 7438.396695] Key type lgssc unregistered [ 7438.697629] LNet: 227568:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7438.705541] LNetError: 227568:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7438.716354] LNet: Removed LNI 192.168.201.141@tcp [ 7439.332552] Key type .llcrypt unregistered [ 7439.334598] Key type ._llcrypt unregistered [ 7463.717531] Key type ._llcrypt registered [ 7463.719994] Key type .llcrypt registered [ 7463.839697] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7465.012361] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7465.029081] alg: No test for adler32 (adler32-zlib) [ 7466.231554] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 7466.485814] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 7468.167275] Key type lgssc registered [ 7469.251511] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7476.437536] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7476.470021] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7477.911938] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7481.895342] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7487.465663] LustreError: 228996:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7487.476539] LustreError: 228996:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7491.523822] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7491.944534] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7497.429156] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7507.390472] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7515.809232] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7524.804096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7525.195896] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7532.108587] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7539.691905] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7547.712335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7554.335426] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7555.978392] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7555.994683] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7556.014401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.033069] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7556.033883] LustreError: 229015:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7558.116428] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7558.126964] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7558.132055] Lustre: Skipped 1 previous similar message [ 7561.257893] Lustre: lustre-OST0000: Not available for connect from 192.168.201.41@tcp (stopping) [ 7563.234955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7563.242512] Lustre: Skipped 1 previous similar message [ 7564.170374] Lustre: server umount lustre-OST0000 complete [ 7576.063118] LustreError: 229015:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7600.355887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7600.734340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7607.210594] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7613.052235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7613.066932] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7613.067052] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7615.468290] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7617.333505] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7623.244567] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ae5465b6800.ost_server_uuid 50 [ 7624.809657] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8ae5465b6800.ost_server_uuid in IDLE state after 0 sec [ 7628.772505] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7628.774658] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7628.774847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7628.785133] Lustre: Skipped 2 previous similar messages [ 7633.896133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7633.914711] Lustre: Skipped 2 previous similar messages [ 7634.886876] Lustre: server umount lustre-OST0000 complete [ 7639.014159] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7644.129306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7644.140909] Lustre: Skipped 3 previous similar messages [ 7644.905143] Lustre: server umount lustre-MDT0000 complete [ 7648.216538] LustreError: 229733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790787245 with bad export cookie 10326250031425659012 [ 7648.223681] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7648.227768] LustreError: 229733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7648.667647] Lustre: server umount lustre-MDT0001 complete [ 7660.558676] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 7663.242875] Key type lgssc unregistered [ 7663.601755] LNet: 234224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7663.616054] LNetError: 234224:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7663.631249] LNet: Removed LNI 192.168.201.141@tcp [ 7664.570388] Key type .llcrypt unregistered [ 7664.573509] Key type ._llcrypt unregistered [ 7699.419871] Key type ._llcrypt registered [ 7699.425671] Key type .llcrypt registered [ 7699.573220] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7700.899583] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7701.045569] alg: No test for adler32 (adler32-zlib) [ 7702.321655] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 7702.639963] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 7704.455777] Key type lgssc registered [ 7705.739570] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7716.604488] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7716.621829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7718.048051] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7722.930678] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7728.611954] LustreError: 235972:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7728.629515] LustreError: 235972:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7731.479513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7731.758773] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7735.905899] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7738.503216] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7744.324398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7744.657440] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7750.785670] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7760.824380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7761.059313] Lustre: lustre-OST0001: new disk, initializing [ 7761.064483] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7761.072816] Lustre: 238277:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7761.202401] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7763.276187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7766.645492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7766.656784] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7766.842661] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7770.060907] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7784.499278] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7792.102114] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7813.103849] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7813.115408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7813.130132] Lustre: Skipped 2 previous similar messages [ 7813.154235] Lustre: Skipped 3 previous similar messages [ 7815.245398] Lustre: server umount lustre-MDT0000 complete [ 7818.221083] LustreError: 238522:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7818.244491] LustreError: 238522:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 7818.892685] LustreError: 235954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790787415 with bad export cookie 13905816708927887038 [ 7818.895661] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7818.909887] LustreError: 235954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7819.228197] Lustre: server umount lustre-MDT0001 complete [ 7832.691938] Lustre: server umount lustre-OST0000 complete [ 7846.082715] Lustre: server umount lustre-OST0001 complete [ 7866.311778] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7874.891160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7875.215673] LustreError: 241389:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7875.300272] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7878.822298] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7880.676196] LustreError: 241390:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7885.801282] LustreError: 241389:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7888.131416] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7888.432702] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7894.304812] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7897.424765] Lustre: 242530:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7905.454345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7905.809118] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7913.000783] LustreError: 242883:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7913.276654] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7920.131239] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7922.584740] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7928.339575] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7929.122810] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7936.983765] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7940.483719] Lustre: 244399:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7953.891255] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7953.894511] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7953.912960] Lustre: Skipped 3 previous similar messages [ 7953.913634] Lustre: Skipped 1 previous similar message [ 7958.033727] LustreError: 244709:0:(ldlm_resource.c:1227:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c796d:0x0:0x0].0x0 (ffff963b6465ab00) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 7958.241973] Lustre: server umount mylustre-MDT0000 complete [ 7959.016201] LustreError: 244166:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7959.043085] LustreError: 244166:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 7961.556346] LustreError: 244402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790787558 with bad export cookie 13905816708927889810 [ 7961.559445] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7961.568763] LustreError: 244402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7961.935043] Lustre: server umount mylustre-MDT0001 complete [ 7976.114161] Lustre: server umount mylustre-OST0000 complete [ 7980.000373] Lustre: 235227:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790787561/real 1790787561] req@ffff963b64725880 x1877776582993664/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790787577 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7980.030232] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7980.047158] Lustre: Skipped 1 previous similar message [ 7980.382348] Lustre: server umount mylustre-OST0001 complete [ 8002.472604] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8012.001917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8012.421911] LustreError: 246711:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8012.478286] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8012.484528] Lustre: Skipped 1 previous similar message [ 8016.566887] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8026.151539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8031.704305] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8034.949118] Lustre: 247854:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8042.086763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8048.877076] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8057.941260] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8060.271054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8066.053663] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 8069.830246] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8077.105522] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8081.401303] Lustre: 249735:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8088.033742] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8088.037677] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8088.044062] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8088.046873] Lustre: Skipped 2 previous similar messages [ 8091.622676] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8094.532280] Lustre: server umount tfs-MDT0000 complete [ 8096.738554] LustreError: 246712:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8096.756874] LustreError: 246712:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 9 previous similar messages [ 8098.307126] LustreError: 246687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790787695 with bad export cookie 13905816708927891700 [ 8098.318844] LustreError: 246687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8098.320192] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8098.748364] Lustre: server umount tfs-MDT0001 complete [ 8113.340356] Lustre: server umount tfs-OST0000 complete [ 8117.219557] Lustre: 235225:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790787698/real 1790787698] req@ffff963b43b3ca80 x1877776583046272/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790787714 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8117.239486] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8117.246470] Lustre: Skipped 3 previous similar messages [ 8118.052551] Lustre: server umount tfs-OST0001 complete [ 8142.576915] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8153.378484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8153.956913] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8153.962808] Lustre: Skipped 3 previous similar messages [ 8159.776531] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8170.196316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8174.536253] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8177.636882] Lustre: 253126:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8184.090981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8189.933530] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8198.785085] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 8198.960783] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8204.306556] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 8206.682087] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8214.033495] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8218.024079] Lustre: 254999:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8224.133360] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 13:03:39 (1790787819) [ 8240.100834] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8240.107932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8240.129943] Lustre: Skipped 2 previous similar messages [ 8240.140775] Lustre: Skipped 6 previous similar messages [ 8245.965623] Lustre: server umount lustre-MDT0000 complete [ 8249.383530] LustreError: 251960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790787846 with bad export cookie 13905816708927893359 [ 8249.396588] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8250.339143] LustreError: 253501:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8250.340446] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8250.379457] LustreError: 253501:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 8250.387871] Lustre: Skipped 5 previous similar messages [ 8256.012148] Lustre: server umount lustre-MDT0001 complete [ 8265.808702] Lustre: server umount lustre-OST0000 complete [ 8275.136594] Lustre: server umount lustre-OST0001 complete [ 8281.281320] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 8290.993482] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8324.324496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8324.589896] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8324.622839] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8324.704220] Lustre: lustre-MDT0000: new disk, initializing [ 8324.755211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8324.762632] Lustre: Skipped 3 previous similar messages [ 8324.770602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8328.397539] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8338.750519] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8338.957680] Lustre: 258179:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8339.005473] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8339.009066] Lustre: Skipped 1 previous similar message [ 8339.101530] Lustre: lustre-MDT0001: new disk, initializing [ 8339.254194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8339.266422] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8344.482301] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8357.385490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8357.876116] Lustre: lustre-OST0000: new disk, initializing [ 8357.881274] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8357.889904] Lustre: 259764:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8359.755507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8359.783149] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8360.011495] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8368.339670] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8381.031680] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8381.357933] Lustre: lustre-OST0001: new disk, initializing [ 8381.364712] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8381.374678] Lustre: 260633:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8382.787745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8382.796172] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8382.933302] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8389.158046] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8402.440510] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 13:06:37 (1790787997) [ 8419.861723] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 13:06:55 (1790788015) [ 8436.705574] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8436.709540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8436.724095] LustreError: Skipped 1 previous similar message [ 8436.727099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8436.741073] Lustre: Skipped 4 previous similar messages [ 8436.755335] Lustre: Skipped 3 previous similar messages [ 8448.991250] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8449.114903] Lustre: server umount lustre-OST0000 complete [ 8459.200114] Lustre: server umount lustre-MDT0000 complete [ 8462.755325] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8462.752974] LustreError: 258172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790788059 with bad export cookie 13905816708927894920 [ 8462.785578] LustreError: 258172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8471.971566] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8471.979316] Lustre: Skipped 9 previous similar messages [ 8477.153032] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8477.529862] Lustre: server umount lustre-MDT0001 complete [ 8489.213570] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8566.145779] Lustre: server umount lustre-OST0001 complete [ 8572.935854] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 8588.833808] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8625.029924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8625.307864] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8625.367860] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8625.466840] Lustre: lustre-MDT0000: new disk, initializing [ 8625.563911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8625.572371] Lustre: Skipped 3 previous similar messages [ 8625.599257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8629.835252] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8642.790312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8642.856211] Lustre: 265490:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8642.887483] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8642.890209] Lustre: Skipped 1 previous similar message [ 8642.935244] Lustre: lustre-MDT0001: new disk, initializing [ 8643.062040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8643.070339] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8649.082977] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8661.064855] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8667.225244] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8674.814722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8675.271550] Lustre: lustre-OST0000: new disk, initializing [ 8675.279948] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8675.286906] Lustre: 267460:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8676.947074] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8676.957661] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8677.104495] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8682.304508] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8691.334501] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8697.838392] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8697.844941] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8697.849715] LustreError: Skipped 1 previous similar message [ 8697.851707] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8697.868296] Lustre: Skipped 5 previous similar messages [ 8697.883950] Lustre: Skipped 2 previous similar messages [ 8702.889823] Lustre: server umount lustre-OST0000 complete [ 8712.390419] Lustre: server umount lustre-MDT0000 complete [ 8713.187752] LustreError: 267548:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8713.219164] LustreError: 267548:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 14 previous similar messages [ 8716.162796] LustreError: 266298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790788313 with bad export cookie 13905816708927896397 [ 8716.165111] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8716.168158] LustreError: 266298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8716.409447] Lustre: server umount lustre-MDT0001 complete [ 8728.066245] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8730.255608] Key type lgssc unregistered [ 8730.523785] LNet: 269220:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8730.534642] LNetError: 269220:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8730.573815] LNet: Removed LNI 192.168.201.141@tcp [ 8731.367224] Key type .llcrypt unregistered [ 8731.370537] Key type ._llcrypt unregistered [ 8762.185035] Key type ._llcrypt registered [ 8762.189923] Key type .llcrypt registered [ 8762.296760] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8763.324510] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8763.390148] alg: No test for adler32 (adler32-zlib) [ 8764.481254] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 8764.763385] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8766.503245] Key type lgssc registered [ 8767.438102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8774.075698] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8774.091779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8775.579262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8780.133225] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8785.891948] LustreError: 270723:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8785.928881] LustreError: 270723:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8789.338719] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8789.586127] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8793.873427] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8801.238456] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8807.566249] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8815.322282] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8815.904359] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8821.058415] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8823.745977] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8832.419349] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8833.497450] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8836.576875] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8836.581695] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8836.593729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8839.652136] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8839.657702] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8839.660202] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8839.679413] Lustre: Skipped 1 previous similar message [ 8842.406655] Lustre: server umount lustre-OST0000 complete [ 8849.892615] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8849.894051] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8849.906613] Lustre: Skipped 1 previous similar message [ 8849.915860] Lustre: Skipped 1 previous similar message [ 8853.290902] Lustre: server umount lustre-MDT0000 complete [ 8855.018659] LustreError: 270723:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8855.035461] LustreError: 270723:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8857.039677] LustreError: 270702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790788454 with bad export cookie 5841641311640903525 [ 8857.064742] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8857.459723] Lustre: server umount lustre-MDT0001 complete [ 8868.995505] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8871.968402] Key type lgssc unregistered [ 8872.263957] LNet: 274306:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8872.270706] LNetError: 274306:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8872.283578] LNet: Removed LNI 192.168.201.141@tcp [ 8873.135131] Key type .llcrypt unregistered [ 8873.137641] Key type ._llcrypt unregistered [ 8890.964167] Key type ._llcrypt registered [ 8890.967506] Key type .llcrypt registered [ 8891.105565] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8892.511346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8892.756975] alg: No test for adler32 (adler32-zlib) [ 8893.907723] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 8894.236606] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8895.983273] Key type lgssc registered [ 8897.724424] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8908.174076] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8908.195339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8923.743435] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8928.863451] LustreError: 275515:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.141@tcp: failed processing log, type 4: rc = -110 [ 8955.551885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8961.941465] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8970.091983] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8972.557821] Lustre: server umount lustre-OST0000 complete [ 8990.836529] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8993.309450] Key type lgssc unregistered [ 8993.541612] LNet: 277148:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8993.548874] LNetError: 277148:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8994.598054] LNet: Removed LNI 192.168.201.141@tcp [ 8995.605085] Key type .llcrypt unregistered [ 8995.607755] Key type ._llcrypt unregistered [ 9034.605075] Key type ._llcrypt registered [ 9034.606673] Key type .llcrypt registered [ 9034.710553] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9035.476365] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9035.615300] alg: No test for adler32 (adler32-zlib) [ 9036.654137] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 9036.883747] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9038.551248] Key type lgssc registered [ 9039.664568] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9050.802159] loop0: detected capacity change from 0 to 204800000 [ 9051.050401] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9051.060995] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9051.556600] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9052.689716] loop0: detected capacity change from 0 to 204800000 [ 9052.820426] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9052.829122] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9053.089938] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9053.847105] loop0: detected capacity change from 0 to 204800000 [ 9054.033868] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9054.044280] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9054.152701] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9054.930958] loop0: detected capacity change from 0 to 204800000 [ 9055.100508] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9056.127838] loop0: detected capacity change from 0 to 204800000 [ 9056.159245] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9060.048799] loop0: detected capacity change from 0 to 204800000 [ 9060.070776] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9064.516326] loop0: detected capacity change from 0 to 204800000 [ 9064.540204] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9067.603624] loop0: detected capacity change from 0 to 204800000 [ 9067.626311] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9071.347895] loop0: detected capacity change from 0 to 204800000 [ 9071.445243] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9071.490712] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9071.607647] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9071.617077] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9071.627612] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9071.636199] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9071.646885] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9071.655816] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9071.663444] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9071.672229] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9071.681264] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9071.698822] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9071.708825] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9071.719897] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9071.737241] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9071.763194] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9077.399562] Lustre: server umount MGS complete [ 9078.179400] loop0: detected capacity change from 0 to 204800000 [ 9078.267723] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9078.695580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9078.763413] Lustre: lustre-MDT0000: Aborting client recovery [ 9079.663574] loop1: detected capacity change from 0 to 204800000 [ 9079.828907] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9079.957593] Lustre: 281006:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9079.963759] Lustre: 281006:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9079.970325] Lustre: 281006:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9079.976557] Lustre: 281006:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9079.982560] Lustre: 281006:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9079.988873] Lustre: 281006:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9079.996271] Lustre: 281006:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9080.028140] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9080.214852] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9080.271052] Lustre: lustre-MDT0001: Aborting client recovery [ 9081.092682] loop2: detected capacity change from 0 to 204800000 [ 9081.296452] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9081.360211] Lustre: 281085:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9081.367538] Lustre: 281085:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9081.378601] Lustre: 281085:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9081.387221] Lustre: 281085:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9081.397263] Lustre: 281085:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9081.406661] Lustre: 281085:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9081.414433] Lustre: 281085:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9081.511070] Lustre: Found index 0 for lustre-OST0000, updating log [ 9081.568918] Lustre: 281103:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9081.687041] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9081.746765] Lustre: lustre-OST0000: Aborting recovery [ 9082.745886] loop3: detected capacity change from 0 to 204800000 [ 9082.891257] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9082.982536] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9083.008782] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9083.028066] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9083.040664] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9083.052461] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9083.065441] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9083.074762] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9083.092567] Lustre: Found index 1 for lustre-OST0001, updating log [ 9083.114404] Lustre: 281174:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9083.174585] Lustre: lustre-OST0001: Aborting recovery [ 9088.320667] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9088.335159] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9088.354577] LustreError: 280950:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9088.393951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9088.405873] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9090.351339] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9090.352402] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9092.261472] LustreError: 280917:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2232us: rc = -2 [ 9092.778600] LustreError: 280917:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 4994us: rc = -2 [ 9092.802879] LustreError: 280917:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 16 previous similar messages [ 9093.812833] LustreError: 280918:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 715us: rc = -2 [ 9093.848419] LustreError: 280918:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 24 previous similar messages [ 9111.013238] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9111.016757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9111.017035] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9111.038489] Lustre: Skipped 3 previous similar messages [ 9111.069913] Lustre: Skipped 3 previous similar messages [ 9112.379323] Lustre: server umount lustre-MDT0000 complete [ 9113.620838] LustreError: 280901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790788710 with bad export cookie 14722112461353695209 [ 9113.626607] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9113.635767] LustreError: 280901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9116.128377] LustreError: 281110:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9116.129111] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9116.159698] LustreError: 281110:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9116.193635] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9116.220162] Lustre: Skipped 1 previous similar message [ 9118.188274] LustreError: 281110:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9118.189719] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9118.211406] Lustre: Skipped 1 previous similar message [ 9120.305804] Lustre: server umount lustre-MDT0001 complete [ 9127.244407] Lustre: server umount lustre-OST0000 complete [ 9134.478143] Lustre: server umount lustre-OST0001 complete [ 9142.912385] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 13:18:58 (1790788738) [ 9165.320125] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9206.505212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9206.751920] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9206.794426] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9206.922066] Lustre: lustre-MDT0000: new disk, initializing [ 9207.019909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9207.026196] Lustre: Skipped 1 previous similar message [ 9207.038381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9211.265932] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9218.256910] Lustre: server umount lustre-MDT0000 complete [ 9238.312622] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 13:20:33 (1790788833) [ 9266.059369] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 9277.714757] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9312.547236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9312.759490] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9312.774908] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9312.776950] Lustre: Skipped 1 previous similar message [ 9312.833980] Lustre: lustre-MDT0000: new disk, initializing [ 9312.887922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9312.897990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9317.373479] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9327.399231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9327.511758] Lustre: 288278:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9327.586643] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9327.588903] Lustre: Skipped 1 previous similar message [ 9327.655289] Lustre: lustre-MDT0001: new disk, initializing [ 9327.767096] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9327.793554] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9327.806077] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9331.596532] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9340.221389] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9346.230355] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9353.249669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9353.504737] Lustre: lustre-OST0000: new disk, initializing [ 9353.508256] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9353.513358] Lustre: 290248:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9353.637680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9360.438083] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9360.941386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9360.954337] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9361.075270] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9371.086621] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9376.683154] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9386.536971] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9390.714748] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9447.913846] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9447.931704] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9447.935431] Lustre: Skipped 2 previous similar messages [ 9447.970398] Lustre: Skipped 1 previous similar message [ 9451.501974] Lustre: server umount lustre-OST0000 complete [ 9458.147338] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9458.165052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9458.171568] Lustre: Skipped 1 previous similar message [ 9458.202932] Lustre: Skipped 1 previous similar message [ 9463.265128] LustreError: 288289:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9463.285866] LustreError: 288289:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 9464.907986] Lustre: server umount lustre-MDT0000 complete [ 9468.396557] LustreError: 288290:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9468.436209] LustreError: 288290:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9469.603102] LustreError: 288271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790789066 with bad export cookie 14722112461353698184 [ 9469.612497] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9469.616743] LustreError: 288271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9470.043939] Lustre: server umount lustre-MDT0001 complete [ 9480.137056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9484.692899] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9490.027054] Lustre: server umount MGS complete [ 9502.280609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9502.790691] LustreError: 294017:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9502.819205] LustreError: 294030:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9502.899214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9507.652071] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9512.418908] LustreError: 294018:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9512.440192] LustreError: 294018:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9517.396257] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9523.410923] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9533.394671] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9539.619759] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9545.771942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9551.869066] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9560.111276] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9588.193375] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9588.200074] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9588.229544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9594.116486] Lustre: server umount lustre-OST0000 complete [ 9598.432750] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9598.446207] Lustre: Skipped 2 previous similar messages [ 9603.677194] Lustre: server umount lustre-MDT0000 complete [ 9607.510602] LustreError: 293999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790789204 with bad export cookie 14722112461353700039 [ 9607.512107] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9607.518672] LustreError: 293999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9607.966200] Lustre: server umount lustre-MDT0001 complete [ 9619.762321] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9622.153307] Key type lgssc unregistered [ 9622.487739] LNet: 297963:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9622.493873] LNetError: 297963:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9622.516523] LNet: Removed LNI 192.168.201.141@tcp [ 9623.458737] Key type .llcrypt unregistered [ 9623.464539] Key type ._llcrypt unregistered [ 9663.232304] Key type ._llcrypt registered [ 9663.235165] Key type .llcrypt registered [ 9663.339710] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 9681.598553] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9682.449887] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9682.542894] alg: No test for adler32 (adler32-zlib) [ 9683.716784] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 9683.926093] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9685.567949] Key type lgssc registered [ 9686.605979] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9723.591060] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9723.638634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9725.047412] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9725.097411] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9725.165233] Lustre: lustre-MDT0000: new disk, initializing [ 9725.264486] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9725.289670] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9730.094585] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9742.108424] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9742.374365] Lustre: 301936:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9742.486265] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9742.491209] Lustre: Skipped 1 previous similar message [ 9742.627807] Lustre: lustre-MDT0001: new disk, initializing [ 9742.868620] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9742.924265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9742.950674] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9750.244631] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9761.457344] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9768.066768] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9775.907847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9776.272923] Lustre: lustre-OST0000: new disk, initializing [ 9776.281715] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9776.289971] Lustre: 303907:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9776.438211] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9783.208916] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9785.390910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9785.399342] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9785.495297] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9794.239576] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9803.509516] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9817.022100] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9829.645711] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9838.782774] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9882.592693] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9882.597421] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9882.615413] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9883.111306] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9883.131386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9883.151347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9883.159140] Lustre: Skipped 1 previous similar message [ 9888.163496] Lustre: server umount lustre-OST0000 complete [ 9893.352310] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9893.355439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9893.368304] Lustre: Skipped 1 previous similar message [ 9898.358436] Lustre: server umount lustre-MDT0000 complete [ 9902.262555] LustreError: 301929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790789499 with bad export cookie 17361939273203411508 [ 9902.264196] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9902.282553] LustreError: 301929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9902.707250] Lustre: server umount lustre-MDT0001 complete [ 9911.420330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9915.650364] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9920.607583] Lustre: server umount MGS complete [ 9934.389340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9934.736455] LustreError: 307876:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9934.757156] LustreError: 307876:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9934.770573] LustreError: 307889:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9934.854467] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9939.949692] LustreError: 307877:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9942.992693] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9944.056643] LustreError: 307876:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9949.156756] LustreError: 307877:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9954.273650] LustreError: 307876:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9957.058078] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9957.701556] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9962.561872] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9970.698520] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9977.286022] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9984.924696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9985.285500] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9991.324906] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9998.113835] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10040.807910] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10040.809385] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10040.828983] Lustre: Skipped 1 previous similar message [10040.845094] Lustre: Skipped 2 previous similar messages [10045.933466] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10045.949606] Lustre: Skipped 1 previous similar message [10046.881790] Lustre: server umount lustre-OST0000 complete [10055.138689] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10055.143622] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10055.158351] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10065.375121] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10065.544580] Lustre: server umount lustre-MDT0000 complete [10066.406688] LustreError: 309103:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10066.442490] LustreError: 309103:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10069.587922] LustreError: 307857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790789666 with bad export cookie 17361939273203413461 [10069.598519] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10069.610559] LustreError: 307857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10070.066247] Lustre: server umount lustre-MDT0001 complete [10084.203739] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [10087.014567] Key type lgssc unregistered [10087.354219] LNet: 311843:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10087.364290] LNetError: 311843:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10088.428617] LNet: Removed LNI 192.168.201.141@tcp [10089.390482] Key type .llcrypt unregistered [10089.391802] Key type ._llcrypt unregistered [10112.658357] Key type ._llcrypt registered [10112.661674] Key type .llcrypt registered [10112.759687] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [10130.759820] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [10131.785675] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10131.891507] alg: No test for adler32 (adler32-zlib) [10133.100310] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [10133.334691] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [10135.023205] Key type lgssc registered [10136.530324] Lustre: Echo OBD driver; http://www.lustre.org/ [10165.239387] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [10181.610119] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [10181.647726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10182.983230] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10183.006680] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10183.073845] Lustre: lustre-MDT0000: new disk, initializing [10183.163292] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10183.178748] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10187.358956] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10198.889673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10199.091306] Lustre: lustre-OST0000: new disk, initializing [10199.094156] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10199.098148] Lustre: Skipped 1 previous similar message [10199.105297] Lustre: 315910:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10199.186233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10200.364330] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10200.382153] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10200.473398] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10205.314627] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11099.959168] Lustre: server umount lustre-MDT0000 complete [11105.335844] LustreError: 315098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790790702 with bad export cookie 11516980106246426997 [11105.359386] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11115.242524] Lustre: server umount lustre-OST0000 complete [11143.166738] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [11155.481488] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [11192.997169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11193.307429] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11193.355429] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11193.440661] Lustre: lustre-MDT0000: new disk, initializing [11193.548563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11193.559661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11198.218349] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11209.361865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11209.552705] Lustre: 320253:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11209.650623] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11209.656216] Lustre: Skipped 1 previous similar message [11209.761615] Lustre: lustre-MDT0001: new disk, initializing [11209.882504] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11209.934061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11209.947063] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11215.074114] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11226.457384] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11233.203192] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11240.749526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11240.994845] Lustre: lustre-OST0000: new disk, initializing [11240.996846] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11240.999974] Lustre: 322222:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11241.051715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11247.136735] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11250.775540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11250.796446] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11250.924157] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11257.847966] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11263.534385] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11263.869465] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11269.670722] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11270.160718] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11276.770024] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11276.771534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11276.778975] LustreError: Skipped 1 previous similar message [11276.780718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11276.821258] Lustre: Skipped 1 previous similar message [11278.652574] Lustre: server umount lustre-OST0000 complete [11287.009825] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11287.017031] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11287.031051] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11287.047326] Lustre: Skipped 2 previous similar messages [11289.733881] Lustre: server umount lustre-MDT0000 complete [11293.890143] LustreError: 321049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790790890 with bad export cookie 11516980106249367816 [11293.900071] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11293.903403] LustreError: 321049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11294.519556] Lustre: server umount lustre-MDT0001 complete [11298.221526] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11300.244877] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 13:54:55 (1790790895) [11307.616362] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11308.357552] LustreError: 324538:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11308.392473] LustreError: 324538:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11308.481706] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11313.016283] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11313.641481] LustreError: 324539:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11318.755420] LustreError: 324538:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11320.528881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11320.883495] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11325.879174] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11334.489458] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11342.387512] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11350.069762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11350.386543] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11357.410279] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11367.121521] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11374.915229] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11375.088317] Lustre: lustre-OST0001: new disk, initializing [11375.091930] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11375.097574] Lustre: 327375:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11375.191942] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11380.921440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11380.941418] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11381.009424] LustreError: 327637:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11382.052469] LustreError: 327637:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11382.064628] LustreError: 327637:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11383.143577] LustreError: 327641:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11383.180120] LustreError: 327641:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11384.307509] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11385.335171] LustreError: 327641:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11385.349983] LustreError: 327641:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11389.479261] LustreError: 327637:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11389.499471] LustreError: 327637:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [11397.793537] LustreError: 327637:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11397.830398] LustreError: 327637:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [11398.111441] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11398.392264] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11405.576846] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ae549c66800.ost_server_uuid 50 [11407.821792] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8ae549c66800.ost_server_uuid in FULL state after 0 sec [11414.183190] LustreError: 327641:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11414.194833] LustreError: 327641:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 30 previous similar messages [11414.936387] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8ae549c66800.ost_server_uuid 50 [11417.352424] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8ae549c66800.ost_server_uuid in IDLE state after 0 sec [11419.462977] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11434.976457] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11434.979188] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11434.986093] LustreError: Skipped 1 previous similar message [11434.990532] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11435.000671] Lustre: Skipped 2 previous similar messages [11435.019575] Lustre: Skipped 1 previous similar message [11437.543437] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11437.558876] Lustre: Skipped 1 previous similar message [11440.020401] Lustre: server umount lustre-OST0001 complete [11442.659861] LustreError: 326360:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11442.670391] LustreError: 326360:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11447.781534] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11447.802120] LustreError: 327593:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11447.805929] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11447.807425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11447.807437] Lustre: Skipped 1 previous similar message [11447.864890] LustreError: 327593:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11453.017654] Lustre: server umount lustre-OST0000 complete [11458.028362] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11458.041567] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11458.056096] Lustre: Skipped 2 previous similar messages [11458.066026] Lustre: Skipped 2 previous similar messages [11464.044920] Lustre: server umount lustre-MDT0000 complete [11468.256761] LustreError: 324539:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11468.280079] LustreError: 324539:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11468.421488] LustreError: 324519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790791065 with bad export cookie 11516980106249368691 [11468.435182] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11468.766431] Lustre: server umount lustre-MDT0001 complete [11480.955478] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [11484.360897] Key type lgssc unregistered [11484.644947] LNet: 330064:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11484.653938] LNetError: 330064:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11485.671377] LNet: Removed LNI 192.168.201.141@tcp [11486.897156] Key type .llcrypt unregistered [11486.899609] Key type ._llcrypt unregistered [11521.530622] Key type ._llcrypt registered [11521.532132] Key type .llcrypt registered [11521.588847] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [11523.163590] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11523.735399] alg: No test for adler32 (adler32-zlib) [11524.911975] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [11525.418417] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [11527.307177] Key type lgssc registered [11529.326403] Lustre: Echo OBD driver; http://www.lustre.org/ [11539.478314] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11539.522727] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11541.313995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11547.183559] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11551.716986] LustreError: 331503:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11551.744900] LustreError: 331503:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11556.842599] LustreError: 331502:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11556.885748] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11561.971079] LustreError: 331503:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11566.056356] LustreError: 331502:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11566.099718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11566.781337] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11572.437867] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11582.293772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11582.530397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11586.657901] LustreError: 333182:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11586.697327] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11590.711524] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11597.301426] LustreError: 333182:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11597.331260] LustreError: 333182:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11599.217790] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11605.911336] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11606.084374] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11611.190636] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11613.004344] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11622.664097] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11631.762829] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11632.027950] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11739.104332] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11739.107324] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11739.131605] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11739.145465] Lustre: Skipped 1 previous similar message [11741.520258] Lustre: server umount lustre-OST0001 complete [11744.237435] LustreError: 333181:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11744.282350] LustreError: 333181:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11750.887229] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11750.889153] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11750.895650] LustreError: Skipped 2 previous similar messages [11750.905065] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11750.924332] Lustre: Skipped 1 previous similar message [11750.929221] Lustre: Skipped 1 previous similar message [11754.466020] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11754.473081] Lustre: Skipped 2 previous similar messages [11759.587281] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11759.603806] Lustre: Skipped 1 previous similar message [11764.191573] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11764.560534] Lustre: server umount lustre-OST0000 complete [11769.824642] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11769.836025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11769.843711] Lustre: Skipped 1 previous similar message [11775.472613] Lustre: server umount lustre-MDT0000 complete [11779.882159] LustreError: 331483:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790791376 with bad export cookie 14139314461624496776 [11779.891987] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11779.901514] LustreError: 331483:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11780.435273] Lustre: server umount lustre-MDT0001 complete [11794.502906] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [11798.237715] Key type lgssc unregistered [11798.514750] LNet: 336462:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11798.519780] LNetError: 336462:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11798.533338] LNet: Removed LNI 192.168.201.141@tcp [11799.703921] Key type .llcrypt unregistered [11799.706397] Key type ._llcrypt unregistered [11857.820423] Key type ._llcrypt registered [11857.824503] Key type .llcrypt registered [11857.952690] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [11859.290541] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11859.317036] alg: No test for adler32 (adler32-zlib) [11860.541520] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [11860.694551] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [11862.295223] Key type lgssc registered [11863.935114] Lustre: Echo OBD driver; http://www.lustre.org/ [11875.744543] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11875.764797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11877.360740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11883.289298] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11887.587873] LustreError: 338767:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11887.598075] LustreError: 338767:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11892.720586] LustreError: 338766:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11895.010275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11895.453430] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11900.478771] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11903.213947] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11911.168855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11911.688891] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11918.491566] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11919.922198] LustreError: 340258:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11921.958815] LustreError: 340260:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11921.980674] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11921.988798] LustreError: 340260:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11927.025544] LustreError: 340260:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11927.046677] LustreError: 340260:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11928.036676] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11928.281250] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11933.711037] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11936.253066] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11947.138813] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11953.655278] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11969.515550] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11969.522352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11969.530982] Lustre: Skipped 2 previous similar messages [11969.542026] Lustre: Skipped 3 previous similar messages [11974.391459] Lustre: server umount lustre-MDT0000 complete [11974.638376] LustreError: 338766:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11979.209098] LustreError: 338748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790791576 with bad export cookie 102177346990216042 [11979.218981] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11979.224021] LustreError: 338748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11979.547663] Lustre: server umount lustre-MDT0001 complete [11994.389473] Lustre: server umount lustre-OST0000 complete [11995.104185] Lustre: 338022:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790791576/real 1790791576] req@ffff963b4179c380 x1877780942883840/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790791592 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11995.140760] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11995.151976] Lustre: Skipped 1 previous similar message [11999.440353] Lustre: server umount lustre-OST0001 complete [12022.391010] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12032.421041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12032.767527] LustreError: 344034:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12032.784622] LustreError: 344034:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12032.822995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12037.584245] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12046.925226] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12047.224443] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12052.313897] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12056.003788] Lustre: 345173:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12064.548386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12064.996987] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12072.111593] LustreError: 345528:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12072.144692] LustreError: 345528:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12072.724725] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12077.226367] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12082.196405] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12087.837652] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12089.425186] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12098.555154] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12103.254728] Lustre: 347045:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12108.264371] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12108.292532] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12108.294309] Lustre: Skipped 1 previous similar message [12108.314612] Lustre: Skipped 2 previous similar messages [12113.395887] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12113.405763] Lustre: Skipped 3 previous similar messages [12114.595937] Lustre: server umount lustre-MDT0000 complete [12120.399502] LustreError: 344014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790791717 with bad export cookie 102177346990217743 [12120.403816] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12120.433637] LustreError: 344014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12120.849345] Lustre: server umount lustre-MDT0001 complete [12135.168478] Lustre: server umount lustre-OST0000 complete [12149.415915] Lustre: server umount lustre-OST0001 complete [12153.486658] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12155.336124] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12156.936794] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 14:09:12 (1790791752) [12182.402594] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12192.548180] loop0: detected capacity change from 0 to 1099511627776 [12193.287312] SGI XFS with ACLs, security attributes, quota, no debug enabled [12193.297812] *** VALIDATE xfs *** [12193.308485] XFS (loop0): Mounting V5 Filesystem [12193.340969] XFS (loop0): Ending clean mount [12194.715100] loop1: detected capacity change from 0 to 18691697672192 [12348.457831] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [12360.906937] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12403.802659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12404.293248] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12404.335058] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12404.460640] Lustre: lustre-MDT0000: new disk, initializing [12404.614480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12404.627161] Lustre: Skipped 1 previous similar message [12404.647161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12409.952838] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12421.432070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12421.576921] Lustre: 352189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12421.622983] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12421.625705] Lustre: Skipped 1 previous similar message [12421.696516] Lustre: lustre-MDT0001: new disk, initializing [12421.873116] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12421.877260] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12427.317102] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12437.649878] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12444.067762] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12451.750636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12452.257208] Lustre: lustre-OST0000: new disk, initializing [12452.275065] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12452.294912] Lustre: 354162:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12453.401888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12453.430330] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12453.594715] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12460.810843] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12473.206727] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12480.031960] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12480.265949] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12485.918773] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12486.278817] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12489.703579] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12489.703962] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12489.708592] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12489.729203] Lustre: Skipped 3 previous similar messages [12494.828464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12494.834162] Lustre: Skipped 2 previous similar messages [12503.007355] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12503.143930] Lustre: server umount lustre-OST0000 complete [12509.155124] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12509.163432] LustreError: Skipped 1 previous similar message [12509.168691] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12509.191570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12509.193503] Lustre: Skipped 2 previous similar messages [12513.848790] Lustre: server umount lustre-MDT0000 complete [12515.297810] LustreError: 352198:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12515.309278] LustreError: 352198:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [12517.742013] LustreError: 352184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790792114 with bad export cookie 102177346990219290 [12517.744795] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12517.757416] LustreError: 352184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12518.049990] Lustre: server umount lustre-MDT0001 complete [12526.830193] XFS (loop0): Unmounting Filesystem [12532.715919] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 14:15:27 (1790792127) [12539.817542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12540.529827] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12540.537817] Lustre: Skipped 2 previous similar messages [12545.535982] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12554.326686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12560.144514] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12568.734552] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12575.661750] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12583.438074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12592.620522] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12602.705119] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12615.137652] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12615.156227] Lustre: Skipped 1 previous similar message [12615.178600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12615.181018] Lustre: Skipped 2 previous similar messages [12621.325835] Lustre: server umount lustre-OST0000 complete [12626.403679] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12632.357913] Lustre: server umount lustre-MDT0000 complete [12638.331501] LustreError: 356706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790792235 with bad export cookie 102177346990220172 [12638.333165] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12638.340521] LustreError: 356706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12639.101586] Lustre: server umount lustre-MDT0001 complete [12655.878114] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [12660.262876] Key type lgssc unregistered [12660.828605] LNet: 360354:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12660.834380] LNetError: 360354:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12660.863946] LNet: Removed LNI 192.168.201.141@tcp [12661.942892] Key type .llcrypt unregistered [12661.944495] Key type ._llcrypt unregistered [12695.955870] Key type ._llcrypt registered [12695.957550] Key type .llcrypt registered [12696.050123] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12697.115822] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12697.375441] alg: No test for adler32 (adler32-zlib) [12698.507865] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [12698.783384] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [12700.519294] Key type lgssc registered [12701.785837] Lustre: Echo OBD driver; http://www.lustre.org/ [12711.425992] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [12711.446040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12713.052483] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12719.341527] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12723.175231] LustreError: 361939:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12723.221897] LustreError: 361939:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12728.291749] LustreError: 361940:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12733.415205] LustreError: 361939:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12733.794153] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12734.620753] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12740.748206] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12750.298599] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12758.553450] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12769.006800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12769.468704] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12778.559445] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12787.741133] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12799.460455] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12799.483575] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12799.490780] Lustre: Skipped 1 previous similar message [12800.025719] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.41@tcp (stopping) [12802.356222] Lustre: server umount lustre-MDT0001 complete [12804.613285] LustreError: 361939:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12804.626555] LustreError: 361939:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12809.702163] LustreError: 362668:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12809.729097] LustreError: 362668:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12816.820229] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12816.887488] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12816.890085] Lustre: Found index 1 for lustre-MDT0001, updating log [12816.924524] Lustre: 361929:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12817.323917] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12822.545135] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12825.201287] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12826.622514] LustreError: 365425:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12826.638061] LustreError: 365425:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12826.647128] LustreError: 361931:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12826.768187] LustreError: 365434:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -17 [12826.779582] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12826.779582] [12837.870669] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12837.879889] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12837.885453] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12837.886364] Lustre: Skipped 2 previous similar messages [12837.913301] Lustre: Skipped 1 previous similar message [12840.977075] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.41@tcp (stopping) [12844.122644] Lustre: server umount lustre-MDT0001 complete [12846.084883] LustreError: 361935:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12846.099530] LustreError: 361935:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [12858.577894] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12858.767278] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12858.776827] Lustre: Found index 1 for lustre-MDT0001, updating log [12858.873570] Lustre: 361929:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12859.364219] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12863.433303] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12864.499925] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12866.607214] LustreError: 366620:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12866.626238] LustreError: 366620:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12866.653199] LustreError: 361931:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12866.659846] LustreError: 361931:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12866.777360] LustreError: 366623:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12866.794539] LustreError: 366623:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12866.801255] LustreError: 366623:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -17 [12866.812728] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.141@tcp [12866.812728] [13171.691796] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13171.700752] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13171.721525] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13171.724026] Lustre: Skipped 2 previous similar messages [13174.759025] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13176.495720] Lustre: server umount lustre-MDT0001 complete [13178.875202] LustreError: 361936:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13178.917507] LustreError: 361936:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [13188.160030] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13188.252439] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13188.257980] Lustre: Found index 1 for lustre-MDT0001, updating log [13188.308923] Lustre: 361929:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13188.753744] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13193.376991] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13194.243521] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13197.327516] LustreError: 374993:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13197.344938] LustreError: 374993:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13197.360138] LustreError: 361931:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13197.364749] LustreError: 361931:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13197.488481] LustreError: 374996:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13197.512874] LustreError: 374996:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -114 [13197.518073] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13197.518073] [13516.259124] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13516.268074] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13516.283290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13516.289043] Lustre: Skipped 3 previous similar messages [13521.732685] Lustre: server umount lustre-MDT0000 complete [13521.902269] LustreError: 368043:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13521.918780] LustreError: 368043:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [13525.417251] LustreError: 361919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790793122 with bad export cookie 12467116792268589743 [13525.445717] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13527.008315] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13527.019991] Lustre: Skipped 2 previous similar messages [13532.061164] Lustre: server umount lustre-MDT0001 complete [13535.849965] Lustre: server umount lustre-OST0000 complete [13545.454916] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [13557.325077] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [13597.520355] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 14:33:12 (1790793192) [13604.587536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13605.025468] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13605.102798] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13605.208816] Lustre: lustre-MDT0000: new disk, initializing [13605.351816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13605.387409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13610.275770] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13621.796589] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13621.926324] Lustre: 385577:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13621.971911] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13621.974937] Lustre: Skipped 1 previous similar message [13622.035856] Lustre: lustre-MDT0001: new disk, initializing [13622.135847] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13622.190939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13622.209651] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13626.557922] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13636.413761] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13642.650941] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13649.574233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13649.922437] Lustre: lustre-OST0000: new disk, initializing [13649.926389] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13649.940217] Lustre: 387543:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13650.058533] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13656.458769] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13658.286915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13658.302088] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13658.468485] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13667.614223] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13676.678481] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13676.691213] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13676.723259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13676.733992] Lustre: Skipped 4 previous similar messages [13683.132237] Lustre: server umount lustre-MDT0000 complete [13684.212696] LustreError: 385588:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13684.254354] LustreError: 385588:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [13687.465171] LustreError: 385570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790793284 with bad export cookie 12467116792268591696 [13687.467816] LustreError: 386382:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13687.469213] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13687.481874] LustreError: 385570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [13687.883301] Lustre: server umount lustre-MDT0001 complete [13703.078024] Lustre: server umount lustre-OST0000 complete [13733.417250] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [13745.979241] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [13776.432255] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13501 sec ========= 14:36:11 (1790793371) [13778.534397] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 14:36:13 (1790793373) === [13782.363658] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 14:36:17 (1790793377) === [13826.154787] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [13828.589129] Key type lgssc unregistered [13828.906700] LNet: 393909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13828.928188] LNetError: 393909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13828.959906] LNet: Removed LNI 192.168.201.141@tcp [13829.824301] Key type .llcrypt unregistered [13829.826174] Key type ._llcrypt unregistered