[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 477523292 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002369] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004013] kvm-guest: setup PV IPIs [ 0.006725] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008011] pid_max: default: 32768 minimum: 301 [ 0.009145] LSM: Security Framework initializing [ 0.010049] Yama: becoming mindful. [ 0.011046] SELinux: Initializing. [ 0.012068] *** VALIDATE selinux *** [ 0.020047] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025076] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026137] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028109] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029100] *** VALIDATE tmpfs *** [ 0.031378] *** VALIDATE proc *** [ 0.032217] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.034259] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035146] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037026] Spectre V2 : User space: Vulnerable [ 0.038009] Speculative Store Bypass: Vulnerable [ 0.040823] debug: unmapping init [mem 0xffffffffafa59000-0xffffffffafa60fff] [ 0.043161] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044653] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045023] ... version: 2 [ 0.046011] ... bit width: 48 [ 0.047010] ... generic registers: 4 [ 0.048010] ... value mask: 0000ffffffffffff [ 0.049014] ... max period: 00007fffffffffff [ 0.050012] ... fixed-purpose events: 3 [ 0.051013] ... event mask: 000000070000000f [ 0.052296] rcu: Hierarchical SRCU implementation. [ 0.054307] smp: Bringing up secondary CPUs ... [ 0.055529] x86: Booting SMP configuration: [ 0.056024] .... node #0, CPUs: #1 #2 #3 [ 0.060157] smp: Brought up 1 node, 4 CPUs [ 0.062013] smpboot: Max logical packages: 1 [ 0.063018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.139804] node 0 deferred pages initialised in 75ms [ 0.144097] devtmpfs: initialized [ 0.145290] x86/mm: Memory block size: 128MB [ 0.148659] gcov: version magic: 0x41383552 [ 0.151236] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.154103] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.158319] pinctrl core: initialized pinctrl subsystem [ 0.161204] [ 0.162008] ************************************************************* [ 0.165015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168013] ** ** [ 0.171016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.174017] ** ** [ 0.177013] ** This means that this kernel is built to expose internal ** [ 0.180015] ** IOMMU data structures, which may compromise security on ** [ 0.183012] ** your system. ** [ 0.186015] ** ** [ 0.189014] ** If you see this message and you are not debugging the ** [ 0.192014] ** kernel, report this immediately to your vendor! ** [ 0.196015] ** ** [ 0.199012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.202015] ************************************************************* [ 0.205690] NET: Registered protocol family 16 [ 0.208456] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.211062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.215070] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.220039] cpuidle: using governor menu [ 0.222572] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.224403] PCI: Using configuration type 1 for base access [ 0.226100] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.233173] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.234020] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.236061] cryptd: max_cpu_qlen set to 1000 [ 0.239003] ACPI: Added _OSI(Module Device) [ 0.240018] ACPI: Added _OSI(Processor Device) [ 0.242016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.244016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.248697] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.255398] ACPI: Interpreter enabled [ 0.257061] ACPI: PM: (supports S0 S3 S4 S5) [ 0.259014] ACPI: Using IOAPIC for interrupt routing [ 0.260078] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.263344] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.272261] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.274034] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.277016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.281063] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.287263] acpiphp: Slot [2] registered [ 0.289105] acpiphp: Slot [5] registered [ 0.292159] acpiphp: Slot [6] registered [ 0.293135] acpiphp: Slot [7] registered [ 0.295131] acpiphp: Slot [8] registered [ 0.296150] acpiphp: Slot [9] registered [ 0.298145] acpiphp: Slot [10] registered [ 0.299153] acpiphp: Slot [3] registered [ 0.301130] acpiphp: Slot [4] registered [ 0.303097] acpiphp: Slot [11] registered [ 0.304094] acpiphp: Slot [12] registered [ 0.306135] acpiphp: Slot [13] registered [ 0.308104] acpiphp: Slot [14] registered [ 0.309115] acpiphp: Slot [15] registered [ 0.311177] acpiphp: Slot [16] registered [ 0.313113] acpiphp: Slot [17] registered [ 0.314093] acpiphp: Slot [18] registered [ 0.316159] acpiphp: Slot [19] registered [ 0.317082] acpiphp: Slot [20] registered [ 0.318061] acpiphp: Slot [21] registered [ 0.319095] acpiphp: Slot [22] registered [ 0.321093] acpiphp: Slot [23] registered [ 0.323132] acpiphp: Slot [24] registered [ 0.325131] acpiphp: Slot [25] registered [ 0.327107] acpiphp: Slot [26] registered [ 0.328113] acpiphp: Slot [27] registered [ 0.330110] acpiphp: Slot [28] registered [ 0.332085] acpiphp: Slot [29] registered [ 0.333104] acpiphp: Slot [30] registered [ 0.335123] acpiphp: Slot [31] registered [ 0.336056] PCI host bridge to bus 0000:00 [ 0.338021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.341024] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.343023] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.346026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.349025] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.352032] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.355195] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.358024] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.362275] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.371791] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.377480] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.380019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.382018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.385024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.388171] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.390832] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.394041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.397646] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.403016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.415017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.422027] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.427000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.432013] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.441016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.458016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.469879] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.476013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.483016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.502022] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.513091] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.527013] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.535012] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.553014] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.564422] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.575016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.582000] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.597017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.607935] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.614015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.620014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.635015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.645070] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.652018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.658015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.679018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.689578] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.692406] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.695329] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.698463] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.699208] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.704257] iommu: Default domain type: Passthrough [ 0.706389] SCSI subsystem initialized [ 0.708140] ACPI: bus type USB registered [ 0.710132] usbcore: registered new interface driver usbfs [ 0.712083] usbcore: registered new interface driver hub [ 0.713095] usbcore: registered new device driver usb [ 0.715182] pps_core: LinuxPPS API ver. 1 registered [ 0.718012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.721089] PTP clock support registered [ 0.724079] EDAC MC: Ver: 3.0.0 [ 0.726102] PCI: Using ACPI for IRQ routing [ 0.728892] NetLabel: Initializing [ 0.730011] NetLabel: domain hash size = 128 [ 0.732013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.734086] NetLabel: unlabeled traffic allowed by default [ 0.738087] vgaarb: loaded [ 0.739521] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.741014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.748636] clocksource: Switched to clocksource kvm-clock [ 0.855395] VFS: Disk quotas dquot_6.6.0 [ 0.857127] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.859725] *** VALIDATE ramfs *** [ 0.860779] *** VALIDATE hugetlbfs *** [ 0.862383] pnp: PnP ACPI init [ 0.865746] pnp: PnP ACPI: found 6 devices [ 0.880779] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.884056] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.886361] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.888543] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.890198] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.891977] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.894017] NET: Registered protocol family 2 [ 0.896089] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.901207] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.905910] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.911181] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.915068] TCP: Hash tables configured (established 65536 bind 65536) [ 0.919732] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.923879] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.927078] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.930358] NET: Registered protocol family 1 [ 0.933341] RPC: Registered named UNIX socket transport module. [ 0.935809] RPC: Registered udp transport module. [ 0.937550] RPC: Registered tcp transport module. [ 0.939549] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.942370] NET: Registered protocol family 44 [ 0.944406] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.946814] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.948944] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.951434] PCI: CLS 0 bytes, default 64 [ 0.953213] Unpacking initramfs... [ 2.344298] debug: unmapping init [mem 0xffff8eedbcc54000-0xffff8eedbffbffff] [ 2.348395] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.351050] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.354582] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.860060] Initialise system trusted keyrings [ 2.861614] Key type blacklist registered [ 2.863756] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.875914] zbud: loaded [ 2.879387] *** VALIDATE nfs *** [ 2.880584] *** VALIDATE nfs4 *** [ 2.882072] pstore: using deflate compression [ 2.885420] Platform Keyring initialized [ 2.994152] NET: Registered protocol family 38 [ 2.996104] Key type asymmetric registered [ 2.997688] Asymmetric key parser 'x509' registered [ 2.999579] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.002783] io scheduler mq-deadline registered [ 3.004720] io scheduler kyber registered [ 3.006561] io scheduler bfq registered [ 3.008886] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.012253] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.015377] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.018514] ACPI: Power Button [PWRF] [ 3.024040] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.031036] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.050334] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.057287] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.072155] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.099593] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.127380] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.132402] Non-volatile memory driver v1.3 [ 3.133875] Linux agpgart interface v0.103 [ 3.163282] virtio_blk virtio1: [vda] 146624 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.166776] vda: detected capacity change from 0 to 75071488 [ 3.190927] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.194278] vdb: detected capacity change from 0 to 1073741824 [ 3.215087] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.218658] vdc: detected capacity change from 0 to 2621440000 [ 3.248435] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.252395] vdd: detected capacity change from 0 to 2621440000 [ 3.272935] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.277238] vde: detected capacity change from 0 to 4294967296 [ 3.297428] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.300749] vdf: detected capacity change from 0 to 4294967296 [ 3.311898] libphy: Fixed MDIO Bus: probed [ 3.317655] usbcore: registered new interface driver usbserial_generic [ 3.320484] usbserial: USB Serial support registered for generic [ 3.322923] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.328198] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.330468] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.333242] mousedev: PS/2 mouse device common for all mice [ 3.335676] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.339171] rtc_cmos 00:05: RTC can wake from S4 [ 3.343498] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.344971] rtc_cmos 00:05: registered as rtc0 [ 3.350658] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.351128] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.357535] intel_pstate: CPU model not supported [ 3.361246] hid: raw HID events driver (C) Jiri Kosina [ 3.363386] usbcore: registered new interface driver usbhid [ 3.365778] usbhid: USB HID core driver [ 3.367644] drop_monitor: Initializing network drop monitor service [ 3.371075] Initializing XFRM netlink socket [ 3.373946] NET: Registered protocol family 10 [ 3.377491] Segment Routing with IPv6 [ 3.379212] NET: Registered protocol family 17 [ 3.382194] mpls_gso: MPLS GSO support [ 3.387206] RAS: Correctable Errors collector initialized. [ 3.388938] AVX version of gcm_enc/dec engaged. [ 3.390743] AES CTR mode by8 optimization enabled [ 3.480505] sched_clock: Marking stable (3480472573, 0)->(4394360959, -913888386) [ 3.485115] registered taskstats version 1 [ 3.487690] Loading compiled-in X.509 certificates [ 3.490308] zswap: loaded using pool lzo/zbud [ 3.519552] Key type big_key registered [ 3.536084] Key type encrypted registered [ 3.538082] ima: No TPM chip found, activating TPM-bypass! [ 3.541341] ima: Allocated hash algorithm: sha1 [ 3.542837] ima: No architecture policies found [ 3.544419] evm: Initialising EVM extended attributes: [ 3.546380] evm: security.selinux [ 3.547939] evm: security.ima [ 3.549045] evm: security.capability [ 3.550288] evm: HMAC attrs: 0x1 [ 3.552554] rtc_cmos 00:05: setting system clock to 2026-08-27 11:54:33 UTC (1787831673) [ 3.559796] debug: unmapping init [mem 0xffffffffb0a03000-0xffffffffb0bfffff] [ 3.562908] debug: unmapping init [mem 0xffffffffaf782000-0xffffffffafa58fff] [ 3.570074] Write protecting the kernel read-only data: 28672k [ 3.574089] debug: unmapping init [mem 0xffffffffade03000-0xffffffffadffffff] [ 3.577344] debug: unmapping init [mem 0xffffffffae714000-0xffffffffae7fffff] [ 3.614933] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.621573] systemd[1]: Detected virtualization kvm. [ 3.623057] systemd[1]: Detected architecture x86-64. [ 3.625400] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.654892] systemd[1]: No hostname configured. [ 3.656696] systemd[1]: Set hostname to . [ 3.659072] random: systemd: uninitialized urandom read (16 bytes read) [ 3.661839] systemd[1]: Initializing machine ID from random generator. [ 3.708091] random: ln: uninitialized urandom read (6 bytes read) [ 3.813129] random: systemd: uninitialized urandom read (16 bytes read) [ 3.815317] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.820325] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.825281] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Journal Service... Starting Setup Virtual Console... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Reached target Timers. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.554280] device-mapper: uevent: version 1.0.3 [ 4.557115] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... [ 4.973778] random: fast init done Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.370948] virtio_net virtio0 ens2: renamed from eth0 [ 5.453248] scsi host0: ata_piix [ 5.541236] scsi host1: ata_piix [ 5.556899] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.559997] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.297556] dracut-initqueue[584]: RTNETLINK answers: File exists [ 10.055864] random: crng init done [ 10.057532] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.694716] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.031700] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.317887] SELinux: Disabled at runtime. [ 12.380875] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.392268] systemd[1]: Detected virtualization kvm. [ 12.397145] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.929668] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.934649] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.940552] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.944303] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.947895] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.959130] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.965749] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-getty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK [ 13.067409] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... Starting Apply Kernel Variables... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target Slices. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.566252] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.878815] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.987425] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.119232] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.130053] EDAC sbridge: Ver: 1.1.2 [ 15.862258] Key type dns_resolver registered [ 16.186567] NFS: Registering the id_resolver key type [ 16.188946] Key type id_resolver registered [ 16.190912] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... Starting Network Manager... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg339-server login: [ 50.591060] hrtimer: interrupt took 19052413 ns [ 73.476553] libcfs: loading out-of-tree module taints kernel. [ 73.554759] Key type ._llcrypt registered [ 73.557017] Key type .llcrypt registered [ 73.692766] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 92.492302] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 94.204459] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 94.230601] alg: No test for adler32 (adler32-zlib) [ 95.777482] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 96.924664] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 98.791235] Key type lgssc registered [ 100.990827] Lustre: Echo OBD driver; http://www.lustre.org/ [ 119.958855] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 162.891666] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 177.418702] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 177.451439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 178.762314] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 178.802380] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 178.960280] Lustre: lustre-MDT0000: new disk, initializing [ 179.086487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 179.106342] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 183.958630] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 197.592663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 197.726097] Lustre: 6516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 197.768491] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 197.782034] Lustre: Skipped 1 previous similar message [ 197.889982] Lustre: lustre-MDT0001: new disk, initializing [ 197.957839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 197.985534] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 198.003778] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 202.633302] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 208.166180] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 218.166660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 218.556540] Lustre: lustre-OST0000: new disk, initializing [ 218.568444] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 218.588326] Lustre: 8454:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 218.663817] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 223.320120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 223.336511] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 223.449068] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 225.209892] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 239.384919] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 239.501497] Lustre: lustre-OST0001: new disk, initializing [ 239.505218] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 239.510245] Lustre: 9529:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 239.572589] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 245.868988] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 247.318601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 247.332867] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 247.371347] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 256.870703] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 262.614504] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 269.197459] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing check_logdir /tmp/testlogs/ [ 274.215635] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing yml_node [ 278.954849] Lustre: DEBUG MARKER: Client: 2.17.57.60 [ 281.742715] Lustre: DEBUG MARKER: MDS: 2.17.57.60 [ 284.274931] Lustre: DEBUG MARKER: OSS: 2.17.57.60 [ 286.008440] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 07:59:14 EDT 2026 [ 302.428466] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 304.200529] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 320.991977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 321.003504] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 321.017419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 324.066394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 324.086539] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 324.102072] Lustre: Skipped 2 previous similar messages [ 326.449866] Lustre: server umount lustre-MDT0000 complete [ 330.353805] LustreError: 6510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832000 with bad export cookie 17232530151112795471 [ 330.358424] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 330.367367] LustreError: 6510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 334.306117] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 334.308303] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 334.330538] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 334.330545] Lustre: Skipped 3 previous similar messages [ 334.358921] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 335.328042] LustreError: 12122:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 335.344338] LustreError: 12122:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 336.720376] Lustre: server umount lustre-MDT0001 complete [ 340.495942] Lustre: server umount lustre-OST0000 complete [ 343.859227] Lustre: server umount lustre-OST0001 complete [ 350.072858] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 357.645986] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 390.718419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 391.010614] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 391.042383] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 391.171610] Lustre: lustre-MDT0000: new disk, initializing [ 391.271083] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 391.295829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 396.192352] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 406.337254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 406.421917] Lustre: 15077:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 406.500521] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 406.506883] Lustre: Skipped 1 previous similar message [ 406.619251] Lustre: lustre-MDT0001: new disk, initializing [ 406.740097] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 406.810595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 406.824374] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 410.854931] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 419.582822] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 424.879230] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 431.376129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 431.726934] Lustre: lustre-OST0000: new disk, initializing [ 431.732144] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 431.741381] Lustre: 17062:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 431.866539] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 433.142637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 433.148943] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 433.276492] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 437.736115] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 446.225896] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 450.487846] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 450.671194] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 454.976161] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 455.163275] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 458.730109] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 458.733578] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 458.755250] Lustre: Skipped 1 previous similar message [ 458.768311] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 458.775374] Lustre: Skipped 3 previous similar messages [ 463.071102] Lustre: server umount lustre-OST0000 complete [ 468.457736] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 468.483921] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 468.504201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 473.026641] Lustre: server umount lustre-MDT0000 complete [ 474.080935] LustreError: 15945:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 474.097403] LustreError: 15945:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 476.444016] LustreError: 15069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832146 with bad export cookie 17232530151112797004 [ 476.450752] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 476.456348] LustreError: 15069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 476.763351] Lustre: server umount lustre-MDT0001 complete [ 482.046914] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:02:30 (1787832150) [ 487.845679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 488.235788] LustreError: 19325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 488.300134] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 492.119070] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 493.540333] LustreError: 19326:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 500.309602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 505.213505] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 512.937932] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 518.552724] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 524.993698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 531.345763] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 538.362784] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 541.468305] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 546.790831] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 546.795818] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 546.808995] Lustre: Skipped 1 previous similar message [ 546.814209] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 546.818713] Lustre: Skipped 1 previous similar message [ 552.506304] Lustre: server umount lustre-MDT0000 complete [ 553.955597] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 553.971555] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 555.406370] LustreError: 21854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832225 with bad export cookie 17232530151112797886 [ 555.411200] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 555.414167] LustreError: 21854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 555.665865] Lustre: server umount lustre-MDT0001 complete [ 569.051880] Lustre: server umount lustre-OST0000 complete [ 587.114104] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 595.567358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 595.921654] LustreError: 23704:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 595.971434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 595.977353] Lustre: Skipped 2 previous similar messages [ 599.971230] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 607.710491] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 612.913436] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 615.864564] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 621.946508] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 628.230750] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 636.601427] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 636.795915] Lustre: lustre-OST0001: new disk, initializing [ 636.799252] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 636.804355] Lustre: 26018:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 638.382282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 638.400548] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 643.353824] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 643.582952] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 653.197543] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 657.104839] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 669.665184] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 669.670630] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 669.683804] Lustre: Skipped 2 previous similar messages [ 669.699465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 669.706708] Lustre: Skipped 3 previous similar messages [ 675.146433] Lustre: server umount lustre-MDT0000 complete [ 676.321850] LustreError: 24433:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 676.353770] LustreError: 24433:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 678.110664] LustreError: 23686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832348 with bad export cookie 17232530151112798733 [ 678.113557] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 678.123613] LustreError: 23686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 678.503506] Lustre: server umount lustre-MDT0001 complete [ 691.775307] Lustre: server umount lustre-OST0000 complete [ 705.215781] Lustre: server umount lustre-OST0001 complete [ 720.003179] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 729.484108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 729.973604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 729.981465] Lustre: Skipped 3 previous similar messages [ 735.067368] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 744.424824] LustreError: 29061:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 744.453295] LustreError: 29061:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 745.834911] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 750.431935] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 753.371481] Lustre: 30190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 759.948713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 766.385236] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 774.609626] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 780.092566] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 787.372247] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 790.851393] Lustre: 32067:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 797.667489] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 797.685607] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 797.697811] Lustre: Skipped 3 previous similar messages [ 797.702924] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 797.708832] Lustre: Skipped 4 previous similar messages [ 802.612795] Lustre: server umount lustre-MDT0000 complete [ 806.123043] LustreError: 30572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832476 with bad export cookie 17232530151112800028 [ 806.124102] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 806.131766] LustreError: 30572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 806.416753] Lustre: server umount lustre-MDT0001 complete [ 819.261497] Lustre: server umount lustre-OST0000 complete [ 832.380899] Lustre: server umount lustre-OST0001 complete [ 842.983023] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:08:31 (1787832511) [ 873.198632] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 883.052890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 883.475553] LustreError: 34869:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 883.504040] LustreError: 34869:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 887.187096] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 894.659853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 899.154966] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 901.668910] Lustre: 36022:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 907.732474] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 914.648738] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 922.707716] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 929.056150] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 936.996711] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 940.471920] Lustre: 37899:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 946.145379] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 946.153326] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 946.163305] Lustre: Skipped 3 previous similar messages [ 946.169283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 946.175523] Lustre: Skipped 4 previous similar messages [ 952.122375] Lustre: server umount lustre-MDT0000 complete [ 955.542471] LustreError: 34850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832625 with bad export cookie 17232530151112801134 [ 955.548678] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 955.554869] LustreError: 34850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 955.900157] Lustre: server umount lustre-MDT0001 complete [ 969.393807] Lustre: server umount lustre-OST0000 complete [ 983.370256] Lustre: server umount lustre-OST0001 complete [ 993.295267] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:11:01 (1787832661) [ 1008.637384] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1019.178753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1019.808494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1019.811819] Lustre: Skipped 7 previous similar messages [ 1024.899878] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1035.179515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1039.108614] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1041.428303] Lustre: 41196:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1047.426631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1053.633854] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1062.070394] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1068.858416] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1077.520400] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1080.892692] Lustre: 43074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1081.675385] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1081.682128] Lustre: Skipped 1 previous similar message [ 1084.903306] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1084.910334] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1084.916548] Lustre: Skipped 5 previous similar messages [ 1084.946873] Lustre: Skipped 7 previous similar messages [ 1090.582879] Lustre: server umount lustre-MDT0000 complete [ 1094.350658] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832764 with bad export cookie 17232530151112802240 [ 1094.356716] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1094.361831] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1100.673519] Lustre: server umount lustre-MDT0001 complete [ 1110.009311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1125.282637] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1130.498133] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1134.693591] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1140.726527] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1140.738719] Lustre: Skipped 1 previous similar message [ 1141.050763] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1149.786215] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1151.449328] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1158.061185] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1159.601916] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1165.794773] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1171.413871] Lustre: server umount lustre-MDT0000 complete [ 1171.436212] LustreError: 43950:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1171.465923] LustreError: 43950:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 34 previous similar messages [ 1174.813461] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832844 with bad export cookie 17232530151112803367 [ 1174.814019] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1174.822615] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1175.196633] Lustre: server umount lustre-MDT0001 complete [ 1189.800763] Lustre: server umount lustre-OST0000 complete [ 1192.735941] Lustre: 3651:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787832846/real 1787832846] req@ffff8eee38cdb0c0 x1874677482284288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787832862 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1193.353690] Lustre: server umount lustre-OST0001 complete [ 1201.390721] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 08:14:30 (1787832870) [ 1219.124457] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1229.308609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1234.327989] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1242.249021] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1247.666943] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1250.538750] Lustre: 48831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1257.258969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1263.755905] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1272.974468] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1280.387794] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1289.786413] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1295.259778] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1309.154050] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1310.533164] Lustre: server umount lustre-MDT0000 complete [ 1314.057476] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832984 with bad export cookie 17232530151112804165 [ 1314.068508] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1314.069759] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1314.401364] Lustre: server umount lustre-MDT0001 complete [ 1328.248918] Lustre: server umount lustre-OST0000 complete [ 1331.182118] Lustre: 3652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787832985/real 1787832985] req@ffff8eee136d5e00 x1874677482339968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787833001 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1331.991570] Lustre: server umount lustre-OST0001 complete [ 1342.199977] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 08:16:50 (1787833010) [ 1344.817878] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1346.949606] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 08:16:55 (1787833015) [ 1359.534518] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 1370.252402] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1398.204383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1398.799203] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1398.974719] Lustre: lustre-MDT0000: new disk, initializing [ 1399.262352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1405.674184] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1416.657510] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1424.374571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1424.655512] Lustre: lustre-OST0000: new disk, initializing [ 1424.658940] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1424.662503] Lustre: Skipped 1 previous similar message [ 1424.679531] Lustre: 54913:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1426.614258] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1426.631641] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1426.684225] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1432.142907] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1442.447136] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1444.489409] Lustre: DEBUG MARKER: create test files [ 1483.232551] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1483.238636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1483.256757] Lustre: Skipped 13 previous similar messages [ 1483.264900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1483.272724] Lustre: Skipped 20 previous similar messages [ 1487.548407] Lustre: server umount lustre-OST0000 complete [ 1492.804648] Lustre: server umount lustre-MDT0000 complete [ 1508.301846] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 1511.674510] Key type lgssc unregistered [ 1511.987328] LNet: 56551:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1512.002690] LNetError: 56551:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1512.030858] LNet: Removed LNI 192.168.203.139@tcp [ 1513.278478] Key type .llcrypt unregistered [ 1513.281421] Key type ._llcrypt unregistered [ 1542.248423] Key type ._llcrypt registered [ 1542.256725] Key type .llcrypt registered [ 1542.395216] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1543.548091] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1543.627777] alg: No test for adler32 (adler32-zlib) [ 1544.704949] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 1545.089536] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 1546.847215] Key type lgssc registered [ 1548.471285] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1557.898147] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1557.916239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1559.621380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1565.506820] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1575.046281] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1582.370836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1582.782651] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1584.858659] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1585.028728] Lustre: 59281:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1585.037992] Lustre: 59281:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1585.043970] Lustre: 59281:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1585.050356] Lustre: 59281:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1585.055262] Lustre: 59281:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1585.059305] Lustre: 59281:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1590.621785] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1599.469812] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1605.580799] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1612.194575] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1619.909881] Lustre: server umount lustre-OST0000 complete [ 1633.429558] Lustre: server umount lustre-MDT0000 complete [ 1648.701321] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 1651.604723] Key type lgssc unregistered [ 1651.920826] LNet: 60896:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1651.936738] LNetError: 60896:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1651.953867] LNet: Removed LNI 192.168.203.139@tcp [ 1652.894142] Key type .llcrypt unregistered [ 1652.897177] Key type ._llcrypt unregistered [ 1690.752229] Key type ._llcrypt registered [ 1690.758552] Key type .llcrypt registered [ 1691.034496] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1693.191869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1693.304082] alg: No test for adler32 (adler32-zlib) [ 1694.781722] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 1695.416879] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 1697.239164] Key type lgssc registered [ 1699.101283] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1709.777784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1709.813905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1711.389186] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1716.983923] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1725.706558] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1733.055427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1733.559766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1736.626252] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1736.708776] Lustre: 63736:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1736.716876] Lustre: 63736:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1736.725396] Lustre: 63736:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1736.730485] Lustre: 63736:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1736.740148] Lustre: 63736:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1736.745020] Lustre: 63736:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1740.639901] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1748.712612] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1754.813976] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1798.116798] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1798.122169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1798.160869] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1799.797993] Lustre: server umount lustre-OST0000 complete [ 1804.550686] Lustre: server umount lustre-MDT0000 complete [ 1817.855357] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 1821.630707] Key type lgssc unregistered [ 1822.047696] LNet: 65322:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1822.071450] LNetError: 65322:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1822.094339] LNet: Removed LNI 192.168.203.139@tcp [ 1822.960492] Key type .llcrypt unregistered [ 1822.963089] Key type ._llcrypt unregistered [ 1844.204667] Key type ._llcrypt registered [ 1844.213452] Key type .llcrypt registered [ 1844.334559] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 1864.449226] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1865.240662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1865.337290] alg: No test for adler32 (adler32-zlib) [ 1866.308693] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 1866.578893] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 1868.327248] Key type lgssc registered [ 1869.666657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1918.250289] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1918.288268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1919.739360] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1919.807418] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1919.976298] Lustre: lustre-MDT0000: new disk, initializing [ 1920.087633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1920.115280] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1925.071637] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1937.476758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1937.687815] Lustre: 68475:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1937.772425] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1937.776442] Lustre: Skipped 1 previous similar message [ 1937.876694] Lustre: lustre-MDT0001: new disk, initializing [ 1938.008217] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1938.076754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1938.091575] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1944.364406] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1956.177772] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1963.406132] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1970.780077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1971.099414] Lustre: lustre-OST0000: new disk, initializing [ 1971.103805] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1971.117321] Lustre: 70449:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1971.220737] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1977.483435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1977.505528] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1977.705491] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1978.598662] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1989.823834] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1995.687687] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1995.983350] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2002.456388] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2002.737195] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2008.549258] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2008.553609] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2008.572259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2008.576446] LustreError: Skipped 1 previous similar message [ 2008.609554] Lustre: Skipped 1 previous similar message [ 2013.679324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2013.692337] Lustre: Skipped 1 previous similar message [ 2018.787203] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2018.803632] Lustre: Skipped 1 previous similar message [ 2019.296808] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2019.703570] Lustre: server umount lustre-OST0000 complete [ 2024.929645] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2024.946158] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2024.974967] Lustre: Skipped 1 previous similar message [ 2024.985177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2029.026314] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2029.030516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2029.070981] Lustre: Skipped 1 previous similar message [ 2030.919217] Lustre: server umount lustre-MDT0000 complete [ 2035.117801] LustreError: 68467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787833705 with bad export cookie 5304388617220645775 [ 2035.124127] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2035.133770] LustreError: 68467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2035.636323] Lustre: server umount lustre-MDT0001 complete [ 2047.315587] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 08:28:35 (1787833715) [ 2060.212461] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2107.984029] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 2119.902407] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2164.416022] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2164.660533] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2164.685615] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2164.776133] Lustre: lustre-MDT0000: new disk, initializing [ 2164.844093] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2164.867764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2169.404245] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2180.876128] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2181.099565] Lustre: 76473:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2181.174699] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2181.185730] Lustre: Skipped 1 previous similar message [ 2181.305326] Lustre: lustre-MDT0001: new disk, initializing [ 2181.416356] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2181.457333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2181.478341] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2186.406241] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2196.916039] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2203.643109] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2210.920630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2211.321661] Lustre: lustre-OST0000: new disk, initializing [ 2211.324841] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2211.330668] Lustre: 78444:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2211.397368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2217.546326] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2218.032760] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2218.041624] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2218.148786] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2228.617480] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2234.465836] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2234.816345] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2241.192619] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2241.550318] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2243.555059] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2243.559890] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2243.578411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2249.868336] Lustre: server umount lustre-OST0000 complete [ 2254.309867] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2254.326559] Lustre: Skipped 1 previous similar message [ 2259.815720] Lustre: server umount lustre-MDT0000 complete [ 2263.404245] LustreError: 76465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787833933 with bad export cookie 5304388617220646657 [ 2263.406253] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2263.418415] LustreError: 76465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2263.838613] Lustre: server umount lustre-MDT0001 complete [ 2272.634646] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 08:32:21 (1787833941) [ 2279.156361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2279.592583] LustreError: 80859:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2279.614196] LustreError: 80859:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2279.655681] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2283.916511] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2285.029824] LustreError: 80861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2289.124818] LustreError: 80859:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2292.599954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2297.787472] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2305.403029] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2311.731483] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2319.556084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2319.975137] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2319.978327] Lustre: Skipped 1 previous similar message [ 2326.794195] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2334.996567] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2339.370873] Lustre: 83533:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 12cfa572-e4c1-4045-bfbb-56852e3f7d7d at adminstrative request [ 2342.911774] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2342.922665] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2352.114992] Lustre: MGS: Client 12cfa572-e4c1-4045-bfbb-56852e3f7d7d (at 0@lo) reconnecting [ 2352.134742] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x499cf957f00fea73 to 0x499cf957f00fed13 [ 2352.148415] Lustre: MGC192.168.203.139@tcp: Connection restored to 0@lo (at 0@lo) [ 2376.400936] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2376.584633] Lustre: lustre-OST0001: new disk, initializing [ 2376.589368] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2376.595842] Lustre: 83925:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2383.577215] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2385.551336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2385.559680] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2385.693303] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2393.924535] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2401.260627] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2401.265268] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2401.272357] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2401.272366] Lustre: Skipped 8 previous similar messages [ 2401.296349] Lustre: Skipped 2 previous similar messages [ 2401.307072] LustreError: Skipped 2 previous similar messages [ 2403.276735] Lustre: server umount lustre-OST0001 complete [ 2406.388112] LustreError: 82684:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2406.402897] LustreError: 82684:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2409.441582] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2411.512146] LustreError: 82682:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2411.531305] LustreError: 82682:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2413.790879] Lustre: server umount lustre-OST0000 complete [ 2421.218561] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2421.227237] LustreError: Skipped 1 previous similar message [ 2421.230440] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2421.243130] Lustre: Skipped 2 previous similar messages [ 2424.490358] Lustre: server umount lustre-MDT0000 complete [ 2426.877802] LustreError: 81863:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2426.898819] LustreError: 81863:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2428.542327] LustreError: 80839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834098 with bad export cookie 5304388617220648211 [ 2428.551480] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2428.558563] LustreError: 80839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2429.080147] Lustre: server umount lustre-MDT0001 complete [ 2438.114210] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 08:35:06 (1787834106) [ 2439.767766] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2441.757659] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 08:35:10 (1787834110) [ 2443.452472] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2445.295517] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 08:35:13 (1787834113) [ 2446.928354] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2448.801843] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 08:35:17 (1787834117) [ 2450.382929] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2452.254369] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 08:35:20 (1787834120) [ 2454.272263] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2461.288296] loop: module loaded [ 2461.310953] loop0: detected capacity change from 0 to 4294967296 [ 2461.366093] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2463.357643] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2467.349804] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2471.960455] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2501.586049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2501.638222] LustreError: 88060:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2501.646760] LustreError: 88060:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2501.656215] LustreError: 88060:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2501.664365] LustreError: 88060:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2521.934140] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 2533.719360] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2574.108823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2574.307677] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2574.335779] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2574.416219] Lustre: lustre-MDT0000: new disk, initializing [ 2574.543743] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2574.547595] Lustre: Skipped 1 previous similar message [ 2574.584360] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2580.559212] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2593.530061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2593.728159] Lustre: 90925:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2594.130204] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2597.682136] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2607.070665] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2613.499543] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2622.460120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2622.742346] Lustre: 92895:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2629.576462] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2631.777854] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2631.917254] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2640.034904] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2645.187382] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2645.458459] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2650.662786] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2650.842715] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2657.763491] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2657.766849] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2657.789881] Lustre: Skipped 1 previous similar message [ 2657.801603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2657.801611] Lustre: Skipped 6 previous similar messages [ 2666.975123] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2667.130856] Lustre: server umount lustre-OST0000 complete [ 2677.717949] Lustre: server umount lustre-MDT0000 complete [ 2678.249986] LustreError: 93531:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2678.279277] LustreError: 93531:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2682.244730] LustreError: 90920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834352 with bad export cookie 5304388617220648827 [ 2682.246980] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2682.255911] LustreError: 90920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2682.598271] Lustre: server umount lustre-MDT0001 complete [ 2691.900555] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 08:39:20 (1787834360) [ 2699.298712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2704.540831] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2713.858393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2714.363557] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2714.378188] Lustre: Skipped 3 previous similar messages [ 2718.927992] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2727.080677] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2732.960683] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2739.801537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2746.443239] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2754.061724] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2759.015882] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2759.208397] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2763.693760] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2763.916875] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2770.458778] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2770.708991] Lustre: lustre-OST0001: new disk, initializing [ 2770.711976] Lustre: Skipped 2 previous similar messages [ 2770.715247] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2770.720316] Lustre: Skipped 3 previous similar messages [ 2770.726852] Lustre: 98682:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2776.695098] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2778.763550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2778.772568] Lustre: Skipped 2 previous similar messages [ 2778.786129] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2778.859044] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2786.873384] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2791.701849] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2792.053596] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2796.361284] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2796.588346] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2815.459189] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2843.311074] Lustre: Failing over lustre-MDT0000 [ 2843.826899] Lustre: server umount lustre-MDT0000 complete [ 2844.815505] LustreError: 95309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2844.834057] LustreError: 95309:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 2845.686497] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2845.721407] Lustre: Skipped 4 previous similar messages [ 2862.047516] Lustre: 66701:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787834515/real 1787834515] req@000000004379d322 x1874679338125056/t0(0) o400->MGC192.168.203.139@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787834531 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2862.097915] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2870.404171] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2870.408677] LDISKFS-fs (dm-0): recovery complete [ 2870.434472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2872.658538] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2875.488799] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2875.489073] Lustre: 95309:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2877.573494] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2877.927883] Lustre: 95331:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2877.934366] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2877.941103] Lustre: 95331:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2878.056521] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2878.383336] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2878.717555] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2878.739180] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2879.063099] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2879.080225] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2879.091298] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2879.099266] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2879.764691] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2879.768489] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2880.087577] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2880.101339] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2881.397679] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2881.417897] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2882.078331] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2882.092962] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2882.426116] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2882.431896] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2885.632326] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2885.642523] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2886.304227] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2886.321313] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2886.647162] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2886.666617] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2893.686184] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2893.714192] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2894.353692] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2894.373952] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2894.695126] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2894.710298] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2909.890535] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2909.913352] Lustre: 101533:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2910.574693] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2910.596264] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2910.920291] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2910.932161] LustreError: 101533:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2935.159188] Lustre: 101533:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2935.176119] Lustre: 101533:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2935.193895] Lustre: 101533:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ca4fe23e-8c67-4913-a072-25dbc73f3dee@192.168.203.39@tcp [ 2935.212607] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2935.217920] Lustre: 101533:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000cf8c17ec x1874679317992064/t0(8589934775) o101->ca4fe23e-8c67-4913-a072-25dbc73f3dee@192.168.203.39@tcp:655/0 lens 592/0 e 0 to 0 dl 1787834615 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2935.265242] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2935.272925] Lustre: lustre-MDT0000: Denying connection for new client ca4fe23e-8c67-4913-a072-25dbc73f3dee (at 192.168.203.39@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2935.304629] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2935.342268] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2935.344115] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2935.346823] Lustre: Skipped 2 previous similar messages [ 2935.390097] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2935.394479] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2949.089839] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2949.104431] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2949.115236] Lustre: Skipped 5 previous similar messages [ 2961.887140] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2961.988838] Lustre: server umount lustre-OST0000 complete [ 2964.968195] LustreError: 97135:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2964.988991] LustreError: 97135:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 41 previous similar messages [ 2980.319674] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2980.505951] Lustre: server umount lustre-OST0001 complete [ 2985.443166] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2985.465664] Lustre: Skipped 6 previous similar messages [ 2999.265465] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2999.473581] Lustre: server umount lustre-MDT0000 complete [ 3003.809464] LustreError: 100783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834673 with bad export cookie 5304388617220700039 [ 3003.829697] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3004.319554] Lustre: server umount lustre-MDT0001 complete [ 3014.010697] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 08:44:42 (1787834682) [ 3021.321743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3021.328552] Lustre: *** cfs_fail_loc=197, val=0*** [ 3036.961316] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3042.079752] LustreError: 103513:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.139@tcp: failed processing log, type 4: rc = -110 [ 3047.253669] Lustre: *** cfs_fail_loc=197, val=0*** [ 3047.257900] Lustre: Skipped 65 previous similar messages [ 3067.807725] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3067.817753] Lustre: Skipped 3 previous similar messages [ 3074.359956] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3081.815585] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3083.950874] Lustre: server umount lustre-OST0000 complete [ 3091.754565] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 08:46:00 (1787834760) [ 3112.439205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3112.709441] LustreError: 105132:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3112.731549] LustreError: 105132:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 3118.408076] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3129.196103] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3132.960838] LustreError: 105163:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3132.987837] LustreError: 105163:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3132.999111] LustreError: 105163:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3133.561814] Lustre: server umount lustre-MDT0000 complete [ 3167.647913] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 3178.164888] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3213.315395] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3213.530046] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3213.561865] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3213.628240] Lustre: lustre-MDT0000: new disk, initializing [ 3213.721380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3218.465206] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3228.846515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3229.008381] Lustre: 109339:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3229.310023] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3233.514558] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3243.788980] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3249.841062] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3257.130796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3257.454750] Lustre: 111307:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3263.732508] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3264.525712] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3273.817422] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3278.262823] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3278.458730] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3282.771335] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3282.919727] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3289.573698] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3289.574646] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3289.597859] LustreError: Skipped 2 previous similar messages [ 3289.598671] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3289.613000] Lustre: Skipped 20 previous similar messages [ 3299.295191] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3299.570496] Lustre: server umount lustre-OST0000 complete [ 3310.047953] Lustre: server umount lustre-MDT0000 complete [ 3313.882954] LustreError: 109331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834983 with bad export cookie 5304388617220710658 [ 3313.891399] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3313.898608] LustreError: 109331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3314.380931] Lustre: server umount lustre-MDT0001 complete [ 3322.955186] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 08:49:51 (1787834991) [ 3331.544629] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 3334.199671] Key type lgssc unregistered [ 3334.492039] LNet: 113797:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3334.504073] LNetError: 113797:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3334.523646] LNet: Removed LNI 192.168.203.139@tcp [ 3335.416766] Key type .llcrypt unregistered [ 3335.419567] Key type ._llcrypt unregistered [ 3352.616925] Key type ._llcrypt registered [ 3352.619365] Key type .llcrypt registered [ 3352.734364] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 3367.809580] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3368.857571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3368.910529] alg: No test for adler32 (adler32-zlib) [ 3370.013713] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 3370.312039] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 3372.040665] Key type lgssc registered [ 3373.129511] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3415.983521] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3416.007740] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3417.281197] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3417.329627] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3417.435730] Lustre: lustre-MDT0000: new disk, initializing [ 3417.601333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3417.630881] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3422.377349] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3433.573101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3433.847965] Lustre: lustre-OST0000: new disk, initializing [ 3433.851552] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3433.856583] Lustre: Skipped 1 previous similar message [ 3433.860750] Lustre: 117976:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3433.934792] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3439.704506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3439.715574] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3439.743458] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3440.131448] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3450.060409] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3450.250206] Lustre: lustre-OST0001: new disk, initializing [ 3450.253323] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3450.259631] Lustre: 118839:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3450.333000] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3456.303289] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3459.198107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3459.220135] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3459.261654] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3469.284510] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3469.299624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3469.302695] Lustre: Skipped 1 previous similar message [ 3469.315531] Lustre: Skipped 1 previous similar message [ 3471.212548] Lustre: server umount lustre-MDT0000 complete [ 3478.334832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3483.269833] LustreError: 117162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835153 with bad export cookie 8485124340748501755 [ 3483.278399] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3483.283501] LustreError: 117162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3483.387163] Lustre: server umount lustre-OST0000 complete [ 3487.124526] Lustre: server umount lustre-OST0001 complete [ 3495.830825] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 08:52:44 (1787835164) [ 3518.133929] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 08:53:06 (1787835186) [ 3528.394690] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3553.285893] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 08:53:41 (1787835221) [ 3593.986715] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 3605.512411] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3645.267830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3645.518180] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3645.561221] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3645.656381] Lustre: lustre-MDT0000: new disk, initializing [ 3645.742494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3645.758814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3650.607502] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3661.944067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3662.102963] Lustre: 126594:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3662.193319] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3662.200989] Lustre: Skipped 1 previous similar message [ 3662.275194] Lustre: lustre-MDT0001: new disk, initializing [ 3662.381067] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3662.418911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3662.433573] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3667.193677] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3676.746743] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3682.569878] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3688.516751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3688.734930] Lustre: lustre-OST0000: new disk, initializing [ 3688.739045] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3688.745214] Lustre: 128567:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3688.833918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3694.652209] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3695.725160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3695.732283] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3695.884608] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3703.667633] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3708.482777] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3708.735745] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3713.537760] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3713.730208] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3716.580694] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3716.595633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3716.609646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3716.620233] Lustre: Skipped 1 previous similar message [ 3721.711323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3721.727036] Lustre: Skipped 1 previous similar message [ 3726.820229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3726.835098] Lustre: Skipped 1 previous similar message [ 3730.399496] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3730.799244] Lustre: server umount lustre-OST0000 complete [ 3737.057407] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3737.060674] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3737.097770] Lustre: Skipped 2 previous similar messages [ 3737.144583] Lustre: Skipped 1 previous similar message [ 3747.299772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3747.302673] Lustre: Skipped 3 previous similar messages [ 3750.367137] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3750.560297] Lustre: server umount lustre-MDT0000 complete [ 3755.148088] LustreError: 127389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835425 with bad export cookie 8485124340748502980 [ 3755.151490] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3755.173548] LustreError: 127389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3755.857477] Lustre: server umount lustre-MDT0001 complete [ 3766.784929] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 08:57:14 (1787835434) [ 3773.449269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3773.914994] LustreError: 130982:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3773.945146] LustreError: 130982:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3773.987275] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3778.383699] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3779.042943] LustreError: 130983:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3783.145121] LustreError: 130982:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3786.987355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3791.427947] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3798.524865] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3804.456692] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3811.209489] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3811.717549] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3811.721291] Lustre: Skipped 1 previous similar message [ 3818.443421] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3826.694661] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3831.840247] Lustre: *** cfs_fail_loc=159, val=0*** [ 3847.339374] Lustre: lustre-MDT0000: Client aac12342-9797-4659-b187-a58bb203320e (at 192.168.203.39@tcp) reconnecting [ 3848.866056] Lustre: *** cfs_fail_loc=159, val=0*** [ 3848.867861] Lustre: Skipped 5 previous similar messages [ 3864.244844] Lustre: lustre-MDT0000: Client aac12342-9797-4659-b187-a58bb203320e (at 192.168.203.39@tcp) reconnecting [ 3870.176859] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3870.181040] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3870.182258] LustreError: Skipped 1 previous similar message [ 3870.191954] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3870.212854] Lustre: Skipped 1 previous similar message [ 3875.718413] Lustre: server umount lustre-OST0000 complete [ 3880.417432] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3880.447610] Lustre: Skipped 1 previous similar message [ 3886.389501] Lustre: server umount lustre-MDT0000 complete [ 3890.085219] LustreError: 130962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835560 with bad export cookie 8485124340748503855 [ 3890.087111] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3890.099169] LustreError: 130962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3890.415461] Lustre: server umount lustre-MDT0001 complete [ 3902.014441] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 3904.771435] Key type lgssc unregistered [ 3905.090593] LNet: 134762:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3905.107152] LNetError: 134762:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3905.134252] LNet: Removed LNI 192.168.203.139@tcp [ 3905.977445] Key type .llcrypt unregistered [ 3905.979447] Key type ._llcrypt unregistered [ 3937.241672] Key type ._llcrypt registered [ 3937.243811] Key type .llcrypt registered [ 3937.328373] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3937.865069] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3937.923484] alg: No test for adler32 (adler32-zlib) [ 3939.178304] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 3939.352751] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 3940.991671] Key type lgssc registered [ 3941.993303] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3949.624904] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3949.653577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3951.337195] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3955.983116] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3961.831348] LustreError: 136178:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3961.859749] LustreError: 136178:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3964.608511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3965.024984] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3969.111855] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3976.940793] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3983.206646] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3989.822890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3990.214519] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3996.354080] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3996.643372] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4005.466229] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4019.304043] Lustre: *** cfs_fail_loc=159, val=0*** [ 4035.808162] Lustre: lustre-MDT0000: Client d9958dbf-631a-4744-9de9-ddcad414a36c (at 192.168.203.39@tcp) reconnecting [ 4038.772357] Lustre: *** cfs_fail_loc=159, val=0*** [ 4054.176075] Lustre: lustre-MDT0001: Client d9958dbf-631a-4744-9de9-ddcad414a36c (at 192.168.203.39@tcp) reconnecting [ 4055.175331] Lustre: *** cfs_fail_loc=159, val=0*** [ 4055.185204] Lustre: Skipped 3 previous similar messages [ 4071.579191] Lustre: lustre-MDT0001: Client d9958dbf-631a-4744-9de9-ddcad414a36c (at 192.168.203.39@tcp) reconnecting [ 4075.147187] Lustre: 139232:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4077.801505] Lustre: *** cfs_fail_loc=159, val=0*** [ 4077.806633] Lustre: Skipped 2 previous similar messages [ 4093.118613] Lustre: lustre-MDT0000: Client 4a95de85-31e7-47d4-af26-23837817c7d1 (at 192.168.203.39@tcp) reconnecting [ 4094.395377] Lustre: *** cfs_fail_loc=159, val=0*** [ 4094.404501] Lustre: Skipped 13 previous similar messages [ 4109.965225] Lustre: lustre-MDT0000: Client 4a95de85-31e7-47d4-af26-23837817c7d1 (at 192.168.203.39@tcp) reconnecting [ 4111.922680] Lustre: 139522:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4119.526342] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4119.538090] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4119.560946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4122.594148] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4122.600669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4122.605078] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4122.641195] Lustre: Skipped 1 previous similar message [ 4124.771837] Lustre: server umount lustre-OST0000 complete [ 4128.483218] Lustre: server umount lustre-MDT0000 complete [ 4128.738236] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4128.750464] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4128.756647] LustreError: 139288:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4132.144725] LustreError: 136160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835802 with bad export cookie 17459017295160663984 [ 4132.155175] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4132.160878] LustreError: 136160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4132.407937] Lustre: server umount lustre-MDT0001 complete [ 4142.921675] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4145.823796] Key type lgssc unregistered [ 4146.172682] LNet: 140580:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4146.198260] LNetError: 140580:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4146.233465] LNet: Removed LNI 192.168.203.139@tcp [ 4147.053509] Key type .llcrypt unregistered [ 4147.056880] Key type ._llcrypt unregistered [ 4171.426544] Key type ._llcrypt registered [ 4171.428453] Key type .llcrypt registered [ 4171.545525] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4172.578889] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4172.715681] alg: No test for adler32 (adler32-zlib) [ 4173.827892] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 4174.089990] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4175.816249] Key type lgssc registered [ 4176.863668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4184.141744] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4184.176566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4185.705411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4190.230774] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4195.814300] LustreError: 141997:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4195.837363] LustreError: 141997:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4199.661246] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4200.036881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4204.122711] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4213.641961] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4220.507308] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4229.515973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4229.807104] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4230.852502] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4236.249614] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4244.163600] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4244.776850] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4251.359870] Lustre: 144581:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4254.989277] Lustre: 144630:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4256.638076] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4257.641215] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4261.881492] Lustre: 144778:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4262.869570] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4263.757100] Lustre: 144876:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4270.049196] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4270.055797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4270.065841] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4270.084920] Lustre: Skipped 1 previous similar message [ 4270.108854] Lustre: Skipped 1 previous similar message [ 4273.844874] Lustre: server umount lustre-OST0000 complete [ 4280.295553] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4280.299663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4280.311781] Lustre: Skipped 1 previous similar message [ 4280.328143] Lustre: Skipped 1 previous similar message [ 4284.176355] Lustre: server umount lustre-MDT0000 complete [ 4285.408506] LustreError: 144610:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4285.427956] LustreError: 144610:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4288.063025] LustreError: 141979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835958 with bad export cookie 7259472545627119260 [ 4288.073363] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4288.077457] LustreError: 141979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4288.424655] Lustre: server umount lustre-MDT0001 complete [ 4299.591146] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4302.665073] Key type lgssc unregistered [ 4302.914787] LNet: 145940:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4302.926923] LNetError: 145940:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4302.958225] LNet: Removed LNI 192.168.203.139@tcp [ 4303.922777] Key type .llcrypt unregistered [ 4303.924984] Key type ._llcrypt unregistered [ 4330.587737] Key type ._llcrypt registered [ 4330.592364] Key type .llcrypt registered [ 4330.746516] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4332.440342] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4332.492317] alg: No test for adler32 (adler32-zlib) [ 4333.594488] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 4333.949134] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4335.743174] Key type lgssc registered [ 4336.958090] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4344.445931] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4344.474677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4346.038794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4350.942990] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4355.565262] LustreError: 147355:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4355.585661] LustreError: 147355:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4359.633421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4360.124359] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4363.804078] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4372.597759] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4378.926717] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4386.281432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4386.743762] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4390.906037] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4393.809266] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4403.072575] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4404.233524] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4410.420310] Lustre: *** cfs_fail_loc=159, val=0*** [ 4426.880384] Lustre: lustre-MDT0000: Client 36d9bb6e-3a8d-4130-af75-1da2eba7f390 (at 192.168.203.39@tcp) reconnecting [ 4434.912842] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4434.917380] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4434.923229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4434.940210] Lustre: Skipped 1 previous similar message [ 4439.662853] Lustre: server umount lustre-OST0000 complete [ 4445.159210] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4445.185766] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4445.196775] Lustre: Skipped 2 previous similar messages [ 4449.772577] Lustre: server umount lustre-MDT0000 complete [ 4450.278915] LustreError: 147351:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4450.298657] LustreError: 147351:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4453.731854] LustreError: 147336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836123 with bad export cookie 8218375433010206903 [ 4453.751165] LustreError: 147336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4453.750661] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4454.261738] Lustre: server umount lustre-MDT0001 complete [ 4465.852445] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4468.936784] Key type lgssc unregistered [ 4469.241526] LNet: 151024:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4469.253337] LNetError: 151024:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4469.276958] LNet: Removed LNI 192.168.203.139@tcp [ 4470.579495] Key type .llcrypt unregistered [ 4470.585097] Key type ._llcrypt unregistered [ 4499.329638] Key type ._llcrypt registered [ 4499.334148] Key type .llcrypt registered [ 4499.537651] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4500.355767] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4500.488686] alg: No test for adler32 (adler32-zlib) [ 4501.666396] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 4501.966817] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4503.703275] Key type lgssc registered [ 4504.935713] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4514.607502] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4514.637576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4516.399415] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4522.131103] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4526.565697] LustreError: 152441:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4526.581977] LustreError: 152441:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4531.689478] LustreError: 152440:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4532.885047] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4533.466725] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4539.337651] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4549.835775] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4557.318843] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4565.251310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4565.657720] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4572.549409] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4579.019146] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4580.313786] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4586.184001] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4605.148847] Lustre: lustre-MDT0001: evicting 33cbc223-3706-4efc-97f3-9b42ecf9c1ed (at 192.168.203.39@tcp) by administrative request [ 4605.172178] Lustre: Skipped 1 previous similar message [ 4635.620385] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4635.626169] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4635.648399] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4637.665420] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4637.669082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4637.698135] Lustre: Skipped 1 previous similar message [ 4640.197263] Lustre: server umount lustre-OST0000 complete [ 4645.859929] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4645.873787] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4645.893332] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4647.905173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4647.907460] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4653.024166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4653.032461] Lustre: Skipped 2 previous similar messages [ 4658.143332] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4658.364774] Lustre: server umount lustre-MDT0000 complete [ 4662.087109] LustreError: 152421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836332 with bad export cookie 15319434860715314715 [ 4662.098637] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4662.103153] LustreError: 152421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4662.497101] Lustre: server umount lustre-MDT0001 complete [ 4675.704188] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4679.224476] Key type lgssc unregistered [ 4679.661883] LNet: 156512:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4679.670795] LNetError: 156512:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4679.693473] LNet: Removed LNI 192.168.203.139@tcp [ 4680.518836] Key type .llcrypt unregistered [ 4680.525320] Key type ._llcrypt unregistered [ 4760.327642] Key type ._llcrypt registered [ 4760.330858] Key type .llcrypt registered [ 4760.448512] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4761.653076] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4761.674634] alg: No test for adler32 (adler32-zlib) [ 4762.888736] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 4763.181547] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4764.973062] Key type lgssc registered [ 4766.577114] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4777.690910] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4777.714793] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4779.418589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4784.205887] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4789.736505] LustreError: 159005:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4789.760786] LustreError: 159005:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4794.352257] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4794.893550] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4800.230462] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4809.746773] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4816.499367] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4824.104151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4824.549595] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4825.605359] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4831.582580] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4839.963424] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4841.058683] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4855.281959] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4855.291255] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4855.301445] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4855.324219] Lustre: Skipped 1 previous similar message [ 4857.695734] Lustre: server umount lustre-OST0000 complete [ 4865.511435] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4865.513548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4865.527706] Lustre: Skipped 1 previous similar message [ 4865.542772] Lustre: Skipped 2 previous similar messages [ 4870.625355] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4870.637974] Lustre: Skipped 1 previous similar message [ 4875.746994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4875.752664] Lustre: Skipped 1 previous similar message [ 4877.279589] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4877.563299] Lustre: server umount lustre-MDT0000 complete [ 4880.865246] LustreError: 159005:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4880.890055] LustreError: 159005:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4882.221423] LustreError: 158987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836552 with bad export cookie 9572744177241388165 [ 4882.230411] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4882.237042] LustreError: 158987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4882.670922] Lustre: server umount lustre-MDT0001 complete [ 4896.675514] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4900.364204] Key type lgssc unregistered [ 4900.740601] LNet: 162556:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4900.747830] LNetError: 162556:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4900.784350] LNet: Removed LNI 192.168.203.139@tcp [ 4901.981101] Key type .llcrypt unregistered [ 4901.983208] Key type ._llcrypt unregistered [ 4927.074422] Key type ._llcrypt registered [ 4927.076572] Key type .llcrypt registered [ 4927.204705] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4928.466789] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4928.631625] alg: No test for adler32 (adler32-zlib) [ 4929.826222] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 4930.078185] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4931.770515] Key type lgssc registered [ 4933.169621] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4950.149394] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 09:16:58 (1787836618) [ 4969.202208] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 4980.636966] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5024.404868] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5024.434242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5025.756807] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5025.797368] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5025.934323] Lustre: lustre-MDT0000: new disk, initializing [ 5026.070149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5026.085955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5031.447447] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5044.563734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5044.735437] Lustre: 166821:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5044.814432] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5044.818150] Lustre: Skipped 1 previous similar message [ 5044.887670] Lustre: lustre-MDT0001: new disk, initializing [ 5045.008353] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5045.054869] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5045.077807] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5050.322437] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5061.445727] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5068.320323] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5076.261484] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5076.584800] Lustre: lustre-OST0000: new disk, initializing [ 5076.596831] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5076.643840] Lustre: 168790:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5076.766810] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5083.723799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5083.742255] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5083.958723] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5084.405614] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5094.683354] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5104.623862] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5104.625486] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5104.635091] LustreError: Skipped 1 previous similar message [ 5104.637354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5104.643029] Lustre: Skipped 1 previous similar message [ 5104.662113] Lustre: Skipped 1 previous similar message [ 5109.729019] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5109.755604] Lustre: Skipped 1 previous similar message [ 5110.947873] Lustre: server umount lustre-OST0000 complete [ 5116.899052] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5116.906225] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5116.917335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5119.969087] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5119.969627] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5120.005888] Lustre: Skipped 1 previous similar message [ 5121.411176] Lustre: server umount lustre-MDT0000 complete [ 5125.372312] LustreError: 168800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836795 with bad export cookie 8794103829042560569 [ 5125.373418] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5125.380874] LustreError: 168800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5125.783213] Lustre: server umount lustre-MDT0001 complete [ 5136.910621] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 5140.160447] Key type lgssc unregistered [ 5140.542113] LNet: 170703:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5140.557967] LNetError: 170703:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5140.595451] LNet: Removed LNI 192.168.203.139@tcp [ 5141.579311] Key type .llcrypt unregistered [ 5141.585748] Key type ._llcrypt unregistered [ 5167.880539] Key type ._llcrypt registered [ 5167.883920] Key type .llcrypt registered [ 5168.019570] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5168.909365] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5169.017170] alg: No test for adler32 (adler32-zlib) [ 5170.022975] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 5170.245058] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 5171.967208] Key type lgssc registered [ 5173.360629] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5181.661503] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5181.681959] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5183.249674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5188.109638] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5193.702432] LustreError: 172118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5193.718626] LustreError: 172118:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5197.712749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5198.276796] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5203.274176] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5211.709388] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5217.596720] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5225.500461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5225.904303] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5229.017295] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5232.776551] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5241.946288] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5248.523194] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5248.553196] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5248.577314] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5248.586776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.602549] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5249.948792] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5249.977792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5249.994039] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5250.008785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.027028] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5251.098777] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5251.135220] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5251.157321] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5251.173099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.181321] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5252.324994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.149596] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5253.155485] Lustre: Skipped 1 previous similar message [ 5253.190386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5253.202711] Lustre: Skipped 1 previous similar message [ 5253.214755] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5253.229219] Lustre: Skipped 1 previous similar message [ 5253.234953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.245697] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5253.253841] Lustre: Skipped 1 previous similar message [ 5254.371947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.524729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.539545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.367602] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5257.374804] Lustre: Skipped 3 previous similar messages [ 5257.385791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5257.404826] Lustre: Skipped 3 previous similar messages [ 5257.412405] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5257.418653] Lustre: Skipped 3 previous similar messages [ 5257.423950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.440754] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5257.451649] Lustre: Skipped 3 previous similar messages [ 5258.371468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.438250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.358898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.341496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.124351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.039607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.882452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.896552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.711314] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5265.717943] Lustre: Skipped 8 previous similar messages [ 5265.733498] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5265.746519] Lustre: Skipped 8 previous similar messages [ 5265.754844] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5265.759825] Lustre: Skipped 8 previous similar messages [ 5265.763372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.775788] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5265.783602] Lustre: Skipped 8 previous similar messages [ 5266.757956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.706766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.845792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.868389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.747926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.722374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.702300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.829904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.886983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.973086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.937427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.159503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.154378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.316707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.164425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.296305] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5282.310973] Lustre: Skipped 15 previous similar messages [ 5282.339149] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5282.350329] Lustre: Skipped 15 previous similar messages [ 5282.354240] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5282.361591] Lustre: Skipped 15 previous similar messages [ 5282.369946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.395120] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5282.403841] Lustre: Skipped 15 previous similar messages [ 5283.585779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.521269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.579182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.408879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.439640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.410979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.677821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.747560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.842564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.799288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.813771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.821439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.879489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.875980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.860318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.890328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.080803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.285203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.395736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.435400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.325915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.281519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.103161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.172092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.255872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.228585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.379666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.405266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.351458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.330226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.355664] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5314.362649] Lustre: Skipped 30 previous similar messages [ 5314.387626] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5314.402613] Lustre: Skipped 30 previous similar messages [ 5314.406131] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5314.411096] Lustre: Skipped 30 previous similar messages [ 5314.414868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.422881] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5314.430950] Lustre: Skipped 30 previous similar messages [ 5315.356994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.310412] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5316.321828] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5316.331444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.274353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.177643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.161820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.107460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.992654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.123817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.190878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.192664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.294983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.265781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.372331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.273521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.187938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.068200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.146586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.179415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.258769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.388423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.333670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.465757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.576383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.680775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.592717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.632753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.671352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.501271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.451982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.567119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.465905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.515550] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5346.531082] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5346.543982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.473326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.620646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.618922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.743506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.821425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.793402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.894931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.114611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.134047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.127967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.117441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.166168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.093838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.154430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.257344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.258432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.324872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.461274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.350453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.340863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.215199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.250788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.315045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.395272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.282567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.421840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.305851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.294970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.128304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.081472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.092717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.186773] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5379.191751] Lustre: Skipped 63 previous similar messages [ 5379.210523] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5379.225483] Lustre: Skipped 63 previous similar messages [ 5379.230883] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5379.238573] Lustre: Skipped 63 previous similar messages [ 5379.250767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.261069] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5379.263899] Lustre: Skipped 63 previous similar messages [ 5380.262233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.527989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.618462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.685028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.766350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.755400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.790426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.631236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.559059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.595312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.685251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.074242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.193646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.154987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.348280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.593314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.721835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.556271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.505660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.455264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.399335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.248494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.193913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.172526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.187380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.074277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.036306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.959286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.106453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.958863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.249312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.465770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.585704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.544455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.536683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.430136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.709933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.798433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.896086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.878985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.990520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.951807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.867075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.812059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.731699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.869731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.915680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.071937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.285835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.348537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.525672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.908664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.183859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.073521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.364018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.335765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.352592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.291394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.268874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.122131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.148448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.205638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.166071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.105246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.997228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.770330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.804642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.641347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.570288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.432178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.415265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.518434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.485279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.374378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.209963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.122233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.881665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.242363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.222337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.369815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.368340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.448960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.297942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.169714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.963482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.914641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.143452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.032254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.329218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.573646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.829637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.339193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.725987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.042376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.948448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.155179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.553223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.776233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.970083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.828855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.087236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.005134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.516524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.917200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.095891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.248712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.471754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.707323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.026256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.770699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.486652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.300026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.139274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.001391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.923979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.746805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.547112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.498467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.333398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.967291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.719871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.973347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.335883] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5507.348022] Lustre: Skipped 122 previous similar messages [ 5507.386299] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5507.401979] Lustre: Skipped 122 previous similar messages [ 5507.410676] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5507.424444] Lustre: Skipped 122 previous similar messages [ 5507.430785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.441420] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5507.461557] Lustre: Skipped 122 previous similar messages [ 5508.686923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.917511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.951217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.808480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.646548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.678278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.971182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.970889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.005051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.196281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.294917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.512546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.530162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.554385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.412869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.546596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.526101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.482041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.405974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.532719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.436701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.369099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.271758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.151903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.083758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.118204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.247614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.265324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.327678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.324230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.327593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.513945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.473690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.683259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.664265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.742887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.811868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.808068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.821851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.931138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.981191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.020370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.922570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.078609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.905540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.898596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.955946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.874565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.927449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.991254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.063598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.041720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.294627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.246272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.283304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.191976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.415693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.260805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.328118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.327913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.368085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.292184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.265392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.291173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.296683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.225331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.302387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.144360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.154609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.272546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.239578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.484049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.595566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.508079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.564888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.435625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.543802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.489563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.414489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.415721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.492590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.430683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.465785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.434425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.429489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.400171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.394282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.251569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.234142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.165892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.034569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.029396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.172867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.171846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.203399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.133620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.162514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.101375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.044533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.932572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.783945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.781840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.712330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.658959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.613565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.638664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.668620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.748336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.786954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.653260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.576390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.504084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.449563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.498557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.411483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.469631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.455897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.474547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.388122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.487644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.637627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.818524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.885399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.154619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.178968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.232220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.372616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.563881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.497257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.386498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.537822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.791693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.671767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.659618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.650414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.604607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.580728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.532381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.422200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.342558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.216237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.111737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.959158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.814635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.630555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.631916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.664391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.684361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.845439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.764857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.755441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.802887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.855747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.873130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.764274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.808561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.769726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.816756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.875553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.865680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.841121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.734176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.624178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.520649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.565091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.433439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.439900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.554605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.597995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.483753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.388826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.326843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.337347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.485853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.309417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.149346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.009471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.900906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.069320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.069677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.865429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.859830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.716632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.092108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.320550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.270168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.767370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.848442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.782312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.341822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.823826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.858089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.760377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.763584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.967925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.993952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.000282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.908924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.869673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.814971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.774965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.633218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.676824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.726347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.706454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.878394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.914946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.939384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.265580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.365137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.340580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.328581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.399832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.399771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.490109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.607354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.630519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.663513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.622583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.656109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.553562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.564413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.539605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.585810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.707986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.613299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.683648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.634671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.685225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.892098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.839612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.806399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.559436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.433451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.309357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.318994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.189492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.237108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.164518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.114115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.996056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.041634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.938378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.739265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.722337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.690076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.706603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.550743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.565322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.483785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.368321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.254140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.055278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.018210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.911837] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5763.917494] Lustre: Skipped 254 previous similar messages [ 5763.938945] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5763.949727] Lustre: Skipped 254 previous similar messages [ 5763.956451] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5763.962803] Lustre: Skipped 254 previous similar messages [ 5763.968988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.978966] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5763.986867] Lustre: Skipped 254 previous similar messages [ 5764.853951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.730888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.726198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.694573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.639843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.486584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.625081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.589968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.540781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.493468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.474755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.437806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.392746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.403147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.374908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.459165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.295833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.385206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.234788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.180570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.125786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.085591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.082040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.100976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.146066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.132448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.096951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.023381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.043434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.802879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.579684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.444275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.368678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.385741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.280644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.118658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.953403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.826012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.767562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.763488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.731226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.586138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.706824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.535921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.563559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.401373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.357709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.260326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.234386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.051562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.967076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.871509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.779093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.694385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.541617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.407337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.368732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.323907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.252656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.114202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.123539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.180415] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 5822.186454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.190746] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 5823.176580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.594357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.746353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.050901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.934391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.902234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.051991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.956544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.008146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.975086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.967647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.745178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.999922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.806103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.088364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.603954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.676677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.662207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.804219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.700166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.801406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.770552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.186142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.390490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.488261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.567889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.683368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.832537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.057241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.223328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.369136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.611536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.547686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.524381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.450794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.726185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.056671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.507272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.616368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.542804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.931397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.825868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.927988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.850560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.006060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.962544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.960221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.902200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.097551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.316763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.368268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.780232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.287271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.449866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.710842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.730313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.647604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.619086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.552091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.358874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.275434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.255746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.302206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.150106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.050877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.961347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.022037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.929375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.958457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.850461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.987982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.951271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.932993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.752331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.773459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.580372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.645485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.761358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.608186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.582573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.501839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.491309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.581325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.478643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.513675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.538889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.575400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.493879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.408972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.232674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.119190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.103324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.107233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.038871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.044035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.068905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.078525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.960494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.773594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.761364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.971279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.990833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.042541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.930850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.859270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.727233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.535437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.563572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.561418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.722570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.787144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.843937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.760675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.605493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.423814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.356870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.230195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.343703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.385514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.292399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.484406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.502820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.498546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.522030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.560564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.686814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.704627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.690966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.691896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.660989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.904581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.112357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.788761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.876965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.850130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.895575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.027973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.285148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.225614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.571564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.727838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.028989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.071518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.231596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.152981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.116923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.243711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.130379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.128798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.049653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.060479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.152696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.356416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.373158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.163254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.232695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.350194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.421533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.406381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.369637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.435634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.487191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.377886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.205482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.277189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.293933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.377671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.437047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.568689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.508650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.347970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.203161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.014206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.006428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.885198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.777935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.659195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.677150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.708522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.692211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.587781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.523440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.766667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.811826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.933712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.924931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.807609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.653794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.674530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.629968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.750569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.743495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.729951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.661923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.555497] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6022.562253] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6022.563648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.450532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.276258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.093860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.921368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.802446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.849619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.784648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.745700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.492968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.361500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.172505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.135318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.008557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.918881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.944638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.790309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.626312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.508535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.383487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.417693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.280375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.265474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.132778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.039707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.016135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.131364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.115667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.192292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.930760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.033680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.885714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.951917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.952119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.938456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.031282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.039147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.128508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.191911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.141432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.087570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.110381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.278887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.169363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.279127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.458672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.483643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.397459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.259543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.984350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.807757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.674076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.564719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.505864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.292912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.364084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.217189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.249183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.159572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.048524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.800744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.839388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.717563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.661181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.529326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.436925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.237784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.182547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.059455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.073416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.124060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.964758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.875627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.785834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.649653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.598247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.529292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.513353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.495622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.487622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.421310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.320899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.335527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.344955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.364749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.286524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.264564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.206759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.165849] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6105.178493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.179370] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6106.072454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.018153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.866045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.808315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.920705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.978156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.985567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.087844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.213732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.230316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.302805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.261253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.258796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.305624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.067869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.970926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.794154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.606970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.474317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.372166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.259561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.235539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.156336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.302093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.415148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.574138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.385170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.252422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.177541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.082680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.002404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.952648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.908308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.722491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.687977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.624424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.849288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.053960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.205255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.172679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.247056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.389806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.286504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.379218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.390832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.342698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.317471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.347952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.271443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.365359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.327304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.374248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.264271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.315032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.317675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.383864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.273877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.114646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.124465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.073517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.967689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.854552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.726365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.605130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.486907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.365491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.230740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.081656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.118706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.017922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.860788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.827856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.865433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.793639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.674868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.642138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.462215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.403889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.384888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.459075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.467747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.488153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.553738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.586182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.614324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.511696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.392870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.252644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.238279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.163712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.058518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.951396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.983211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.946967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.019273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.026126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.043815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.244919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.586630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.665622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.768815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.841511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.051927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.057856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.095100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.955072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.051901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.267329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.281934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.267153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.130839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.321576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.295707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.399618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.567193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.707644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.752696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.902244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.830117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.040808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.010617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.043504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.170115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.197384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.311537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.457394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.855172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.990270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.135026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.317844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.368679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.566904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.533911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.642530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.647895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.629336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.685652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.748366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.863136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.011971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.057988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.221333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.290508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.474559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.489691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.573654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.456212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.374758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.992236] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7445.998285] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7446.002584] Lustre: Skipped 494 previous similar messages [ 7451.113495] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7451.123494] Lustre: Skipped 2 previous similar messages [ 7452.066238] Lustre: server umount lustre-OST0000 complete [ 7456.229613] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7456.231159] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7456.238400] Lustre: Skipped 1 previous similar message [ 7461.349503] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7461.355087] Lustre: Skipped 1 previous similar message [ 7461.954492] Lustre: server umount lustre-MDT0000 complete [ 7465.249578] LustreError: 172098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839135 with bad export cookie 6734074216520715103 [ 7465.251597] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7465.268063] LustreError: 172098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7465.618282] Lustre: server umount lustre-MDT0001 complete [ 7475.938484] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 7478.496038] Key type lgssc unregistered [ 7478.818258] LNet: 227401:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7478.828300] LNetError: 227401:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7478.845580] LNet: Removed LNI 192.168.203.139@tcp [ 7479.670750] Key type .llcrypt unregistered [ 7479.676517] Key type ._llcrypt unregistered [ 7503.833331] Key type ._llcrypt registered [ 7503.835434] Key type .llcrypt registered [ 7503.925976] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 7504.647684] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7504.815741] alg: No test for adler32 (adler32-zlib) [ 7505.808236] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 7506.051642] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 7507.720282] Key type lgssc registered [ 7508.663813] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7516.009671] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7516.033899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7517.513823] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7521.228497] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7527.907932] LustreError: 228818:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7527.933374] LustreError: 228818:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7529.923112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7530.350387] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7535.280859] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7542.575523] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7547.508954] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7554.529858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7554.882209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7560.173183] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7563.083626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7567.349549] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7573.456573] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7575.176884] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7575.198837] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7575.206084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7575.214301] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7575.214360] LustreError: 228838:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7578.597498] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7578.610175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7578.615643] Lustre: Skipped 1 previous similar message [ 7580.287452] Lustre: lustre-OST0000: Not available for connect from 192.168.203.39@tcp (stopping) [ 7583.506173] Lustre: server umount lustre-OST0000 complete [ 7595.289440] LustreError: 228838:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7618.935827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7619.481105] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7620.523692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7620.539766] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7620.548609] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7620.561604] Lustre: Skipped 1 previous similar message [ 7626.302914] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7633.764198] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7635.700460] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7642.111359] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b8784e21800.ost_server_uuid 50 [ 7643.880158] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b8784e21800.ost_server_uuid in IDLE state after 0 sec [ 7649.248902] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7649.263549] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7649.289177] Lustre: Skipped 1 previous similar message [ 7649.308639] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7651.299830] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7651.346664] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7651.356547] Lustre: Skipped 1 previous similar message [ 7654.537667] Lustre: server umount lustre-OST0000 complete [ 7658.469639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7658.484610] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7658.507158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7664.567979] Lustre: server umount lustre-MDT0000 complete [ 7666.659865] LustreError: 229824:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7666.675508] LustreError: 229824:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7668.147761] LustreError: 228798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839338 with bad export cookie 5862348550691528136 [ 7668.149573] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7668.159724] LustreError: 228798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7668.486469] Lustre: server umount lustre-MDT0001 complete [ 7678.940928] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 7681.642975] Key type lgssc unregistered [ 7681.981716] LNet: 234040:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7681.996918] LNetError: 234040:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7683.049437] LNet: Removed LNI 192.168.203.139@tcp [ 7683.887335] Key type .llcrypt unregistered [ 7683.890110] Key type ._llcrypt unregistered [ 7713.585956] Key type ._llcrypt registered [ 7713.589313] Key type .llcrypt registered [ 7713.713639] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 7714.621567] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7714.670090] alg: No test for adler32 (adler32-zlib) [ 7715.887623] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 7716.212631] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 7717.895224] Key type lgssc registered [ 7719.123799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7728.370299] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7728.399302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7729.912940] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7733.789554] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7739.371082] LustreError: 235766:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7739.394181] LustreError: 235766:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7742.197469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7742.526387] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7746.894305] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7749.438319] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7755.808984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7756.122438] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7762.350433] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7772.800885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7773.163876] Lustre: lustre-OST0001: new disk, initializing [ 7773.169592] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7773.180668] Lustre: 238072:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7773.374055] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7774.484626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7782.553710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7782.573042] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7782.681856] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7783.423425] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7795.847937] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7802.409839] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7819.232144] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7819.239343] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7819.255963] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7821.798261] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7821.799742] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7821.822634] Lustre: Skipped 2 previous similar messages [ 7821.836138] Lustre: Skipped 3 previous similar messages [ 7824.526584] Lustre: server umount lustre-MDT0000 complete [ 7826.917552] LustreError: 235762:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7826.940612] LustreError: 235762:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7827.614623] LustreError: 235749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839497 with bad export cookie 7944446535904614619 [ 7827.627940] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7827.631331] LustreError: 235749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7827.937535] Lustre: server umount lustre-MDT0001 complete [ 7841.417933] Lustre: server umount lustre-OST0000 complete [ 7855.241507] Lustre: server umount lustre-OST0001 complete [ 7879.888824] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 7890.644412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7891.089653] LustreError: 241191:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7891.182429] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7895.575148] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7896.556445] LustreError: 241192:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7901.680902] LustreError: 241191:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7905.602506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7906.242183] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7911.603625] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7915.205745] Lustre: 242333:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7922.479060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7922.808675] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7929.553164] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7931.024509] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7932.021286] LustreError: 242687:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7932.061756] LustreError: 242687:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7937.150892] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7938.405151] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7945.136703] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 7952.559690] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7956.114945] Lustre: 244205:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7967.712839] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7967.724329] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7967.757479] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7974.372555] LustreError: 241191:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7974.388322] LustreError: 241191:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7974.562262] Lustre: server umount mylustre-MDT0000 complete [ 7977.839096] LustreError: 241167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839647 with bad export cookie 7944446535904617482 [ 7977.844435] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7977.846233] LustreError: 241167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7978.191860] Lustre: server umount mylustre-MDT0001 complete [ 7991.371419] Lustre: server umount mylustre-OST0000 complete [ 8004.272841] Lustre: server umount mylustre-OST0001 complete [ 8025.525929] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 8034.640030] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8034.981621] LustreError: 246513:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8035.044381] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8035.049290] Lustre: Skipped 1 previous similar message [ 8039.280131] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8047.705950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8053.758978] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8056.894715] Lustre: 247655:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8064.338249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8071.671681] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8073.073695] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 8079.342805] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8080.866971] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8086.526716] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 8086.710800] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8096.510990] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8100.248533] Lustre: 249527:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8109.536523] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8109.543862] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8109.550809] Lustre: Skipped 3 previous similar messages [ 8109.554816] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8109.562220] Lustre: Skipped 4 previous similar messages [ 8114.979898] Lustre: server umount tfs-MDT0000 complete [ 8117.223647] LustreError: 249777:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8117.248647] LustreError: 249777:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8118.040937] LustreError: 246489:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839787 with bad export cookie 7944446535904619435 [ 8118.048658] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8118.303566] Lustre: server umount tfs-MDT0001 complete [ 8131.672778] Lustre: server umount tfs-OST0000 complete [ 8144.478542] Lustre: server umount tfs-OST0001 complete [ 8164.415649] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 8174.213170] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8174.856105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8174.868378] Lustre: Skipped 3 previous similar messages [ 8178.677422] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8186.613873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8190.513565] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8193.259777] Lustre: 252923:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8199.455798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8204.796799] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8205.933908] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8211.970888] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 8213.708479] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8219.166801] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8221.424500] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8230.168688] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8235.312259] Lustre: 254793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8242.252690] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:11:50 (1787839910) [ 8258.529271] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8258.543055] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8258.563829] Lustre: Skipped 3 previous similar messages [ 8258.569213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8258.571927] Lustre: Skipped 4 previous similar messages [ 8263.971305] Lustre: server umount lustre-MDT0000 complete [ 8265.187100] LustreError: 251780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8265.209239] LustreError: 251780:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 8266.912504] LustreError: 252925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839936 with bad export cookie 7944446535904621346 [ 8266.917805] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8266.923527] LustreError: 252925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 8270.304772] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8270.310243] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8270.327631] Lustre: Skipped 4 previous similar messages [ 8270.331361] Lustre: Skipped 4 previous similar messages [ 8273.315486] Lustre: server umount lustre-MDT0001 complete [ 8291.295131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8291.427370] Lustre: server umount lustre-OST0000 complete [ 8309.219852] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8309.399975] Lustre: server umount lustre-OST0001 complete [ 8316.345329] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 8326.384730] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 8363.693282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8363.968805] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8364.005772] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8364.120374] Lustre: lustre-MDT0000: new disk, initializing [ 8364.200637] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8364.207295] Lustre: Skipped 3 previous similar messages [ 8364.221418] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8368.756447] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8379.063393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8379.233470] Lustre: 257977:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8379.341715] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8379.345680] Lustre: Skipped 1 previous similar message [ 8379.455804] Lustre: lustre-MDT0001: new disk, initializing [ 8379.679630] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8379.695879] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8384.734555] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8394.900470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8395.198416] Lustre: lustre-OST0000: new disk, initializing [ 8395.204129] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8395.214343] Lustre: 259558:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8396.913244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8396.925819] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8397.052080] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8401.349751] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8411.391724] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8411.662443] Lustre: lustre-OST0001: new disk, initializing [ 8411.666948] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8411.684628] Lustre: 260429:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8413.276767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8413.287550] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8413.420540] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8417.911628] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8430.608375] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:14:59 (1787840099) [ 8446.499384] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:15:14 (1787840114) [ 8463.842104] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8463.844563] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8463.849732] LustreError: Skipped 1 previous similar message [ 8463.858897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8463.886526] Lustre: Skipped 1 previous similar message [ 8463.887395] Lustre: Skipped 2 previous similar messages [ 8467.884515] Lustre: server umount lustre-OST0000 complete [ 8478.520238] Lustre: server umount lustre-MDT0000 complete [ 8481.969123] LustreError: 257970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787840151 with bad export cookie 7944446535904623131 [ 8481.978580] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8481.980021] LustreError: 257970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8488.487922] Lustre: server umount lustre-MDT0001 complete [ 8501.343907] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 8573.895714] Lustre: server umount lustre-OST0001 complete [ 8580.283986] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 8595.600715] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 8632.700602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8632.884576] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8632.910379] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8633.000883] Lustre: lustre-MDT0000: new disk, initializing [ 8633.109290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8633.114783] Lustre: Skipped 3 previous similar messages [ 8633.142507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8637.902904] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8648.675491] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8648.903210] Lustre: 265276:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8648.956750] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8648.960223] Lustre: Skipped 1 previous similar message [ 8649.083788] Lustre: lustre-MDT0001: new disk, initializing [ 8649.323754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8649.344623] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8655.534270] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8665.860767] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8671.365338] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8678.906837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8679.207104] Lustre: 267245:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8680.591338] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8680.725806] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8685.677171] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8694.530479] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8706.533288] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8706.537253] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8706.541658] Lustre: Skipped 4 previous similar messages [ 8706.543034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8706.543040] Lustre: Skipped 8 previous similar messages [ 8709.066946] Lustre: server umount lustre-OST0000 complete [ 8719.299070] Lustre: server umount lustre-MDT0000 complete [ 8721.890293] LustreError: 267965:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8721.908480] LustreError: 267965:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 8722.579788] LustreError: 265269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787840392 with bad export cookie 7944446535904624601 [ 8722.590361] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8722.594321] LustreError: 265269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8722.851630] Lustre: server umount lustre-MDT0001 complete [ 8732.812231] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 8735.385872] Key type lgssc unregistered [ 8735.693572] LNet: 269008:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8735.707548] LNetError: 269008:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8735.734970] LNet: Removed LNI 192.168.203.139@tcp [ 8736.819158] Key type .llcrypt unregistered [ 8736.820862] Key type ._llcrypt unregistered [ 8762.805693] Key type ._llcrypt registered [ 8762.807428] Key type .llcrypt registered [ 8762.894251] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 8763.785052] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8763.812256] alg: No test for adler32 (adler32-zlib) [ 8764.852943] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 8765.105533] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 8766.785072] Key type lgssc registered [ 8768.157688] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8776.184223] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8776.230827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8778.204054] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8783.617802] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8788.449348] LustreError: 270471:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8788.461706] LustreError: 270471:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8793.510243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8793.571158] LustreError: 270472:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8794.166275] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8801.019300] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8809.898839] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8816.262265] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8823.948038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8824.277960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8830.609978] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8837.887577] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8839.103989] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8843.750794] LustreError: lustre-OST0000-osc-MDT0001: operation ost_create to node 0@lo failed: rc = -107 [ 8843.755046] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8843.759945] LustreError: Skipped 1 previous similar message [ 8843.789089] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8847.570801] Lustre: server umount lustre-OST0000 complete [ 8853.923673] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8853.930912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8853.934517] Lustre: Skipped 1 previous similar message [ 8853.954680] Lustre: Skipped 1 previous similar message [ 8856.955725] Lustre: server umount lustre-MDT0000 complete [ 8859.104156] LustreError: 270468:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8859.134980] LustreError: 270468:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8860.310742] LustreError: 270452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787840530 with bad export cookie 9920652711084653079 [ 8860.318393] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8860.318859] LustreError: 271202:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0001: can't precreate: rc = -5 [ 8860.318931] LustreError: 271202:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0001: cannot precreate objects: rc = -5 [ 8860.328100] LustreError: 270452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8860.649852] Lustre: server umount lustre-MDT0001 complete [ 8872.019976] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 8874.766217] Key type lgssc unregistered [ 8875.058534] LNet: 274060:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8875.065166] LNetError: 274060:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8875.080217] LNet: Removed LNI 192.168.203.139@tcp [ 8876.044531] Key type .llcrypt unregistered [ 8876.046079] Key type ._llcrypt unregistered [ 8892.065180] Key type ._llcrypt registered [ 8892.067529] Key type .llcrypt registered [ 8892.178266] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 8893.206446] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8893.233055] alg: No test for adler32 (adler32-zlib) [ 8894.286955] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 8894.589111] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 8896.287157] Key type lgssc registered [ 8897.416498] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8905.120358] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8905.148197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8920.671944] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8925.792113] LustreError: 275255:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.139@tcp: failed processing log, type 4: rc = -110 [ 8952.480823] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8958.717753] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 8966.664429] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8969.081831] Lustre: server umount lustre-OST0000 complete [ 8984.292350] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 8986.682290] Key type lgssc unregistered [ 8986.896929] LNet: 276888:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8986.906300] LNetError: 276888:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8986.917514] LNet: Removed LNI 192.168.203.139@tcp [ 8987.449165] Key type .llcrypt unregistered [ 8987.451091] Key type ._llcrypt unregistered [ 9029.809216] Key type ._llcrypt registered [ 9029.811085] Key type .llcrypt registered [ 9029.897368] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 9030.615806] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9030.718996] alg: No test for adler32 (adler32-zlib) [ 9031.697329] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 9031.949702] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 9033.639408] Key type lgssc registered [ 9034.679825] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9046.403812] loop0: detected capacity change from 0 to 204800000 [ 9046.537614] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9046.549961] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9046.853417] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9047.830738] loop0: detected capacity change from 0 to 204800000 [ 9048.015582] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9048.024872] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9048.391396] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9049.333767] loop0: detected capacity change from 0 to 204800000 [ 9049.477779] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9049.493335] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9049.578602] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9050.436834] loop0: detected capacity change from 0 to 204800000 [ 9050.621885] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9051.781189] loop0: detected capacity change from 0 to 204800000 [ 9051.804803] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9055.702889] loop0: detected capacity change from 0 to 204800000 [ 9055.729491] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9059.367412] loop0: detected capacity change from 0 to 204800000 [ 9059.392454] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9062.861875] loop0: detected capacity change from 0 to 204800000 [ 9062.886734] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9066.528858] loop0: detected capacity change from 0 to 204800000 [ 9066.607842] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9066.627369] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9066.723594] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9066.733388] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9066.749237] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9066.770962] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9066.781234] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9066.790903] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9066.807634] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9066.815921] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9066.827260] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9066.837959] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9066.847874] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9066.859352] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9066.870889] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9066.885769] Lustre: 280307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9072.381335] Lustre: server umount MGS complete [ 9073.198346] loop0: detected capacity change from 0 to 204800000 [ 9073.300462] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9073.868485] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9073.911596] Lustre: lustre-MDT0000: Aborting client recovery [ 9074.993825] loop1: detected capacity change from 0 to 204800000 [ 9075.134627] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9075.201925] Lustre: 280741:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9075.216338] Lustre: 280741:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9075.226230] Lustre: 280741:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9075.237161] Lustre: 280741:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9075.246201] Lustre: 280741:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9075.256962] Lustre: 280741:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9075.272438] Lustre: 280741:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9075.287740] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9075.478630] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9075.541277] Lustre: lustre-MDT0001: Aborting client recovery [ 9076.648712] loop2: detected capacity change from 0 to 204800000 [ 9076.888628] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9076.991563] Lustre: 280820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9077.011307] Lustre: 280820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9077.022179] Lustre: 280820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9077.041984] Lustre: 280820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9077.055265] Lustre: 280820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9077.074353] Lustre: 280820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9077.084738] Lustre: 280820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9077.251268] Lustre: Found index 0 for lustre-OST0000, updating log [ 9077.302441] Lustre: 280837:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9077.382975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9077.398888] Lustre: lustre-OST0000: Aborting recovery [ 9078.265111] loop3: detected capacity change from 0 to 204800000 [ 9078.413830] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9078.455088] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9078.461526] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9078.467618] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9078.473467] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9078.484230] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9078.494572] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9078.501260] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9078.521082] Lustre: Found index 1 for lustre-OST0001, updating log [ 9078.541731] Lustre: 280907:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9078.626091] Lustre: lustre-OST0001: Aborting recovery [ 9083.773818] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9083.783068] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9083.802911] LustreError: 280684:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9083.817392] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9083.847655] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9085.797477] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9085.798986] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9087.091617] LustreError: 280651:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 6393us: rc = -2 [ 9087.622810] LustreError: 280651:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1519us: rc = -2 [ 9087.642677] LustreError: 280651:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 9101.280649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9101.280893] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9101.281685] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9101.292507] Lustre: Skipped 1 previous similar message [ 9106.401045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9106.410808] Lustre: Skipped 6 previous similar messages [ 9106.508328] Lustre: server umount lustre-MDT0000 complete [ 9107.361523] LustreError: 280635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787840777 with bad export cookie 12504274400479828381 [ 9107.376039] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9111.520708] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9111.522344] LustreError: 280649:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9111.531483] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9111.538556] Lustre: Skipped 3 previous similar messages [ 9111.596931] LustreError: 280649:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9112.544709] LustreError: 280648:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9112.571486] LustreError: 280648:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9114.052087] Lustre: server umount lustre-MDT0001 complete [ 9121.656066] Lustre: server umount lustre-OST0000 complete [ 9129.912059] Lustre: server umount lustre-OST0001 complete [ 9142.466673] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 10:26:49 (1787840809) [ 9173.191911] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 9186.031089] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 9227.063643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9227.545606] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9227.577160] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9227.704731] Lustre: lustre-MDT0000: new disk, initializing [ 9227.823197] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9227.836946] Lustre: Skipped 1 previous similar message [ 9227.885214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9232.970968] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9245.213388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9245.289748] Lustre: 284983:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9245.330584] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9245.333769] Lustre: Skipped 1 previous similar message [ 9245.384356] Lustre: lustre-MDT0001: new disk, initializing [ 9245.481747] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9245.524362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9245.534534] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9248.988098] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9258.898574] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9266.162033] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9273.193257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9273.570123] Lustre: lustre-OST0000: new disk, initializing [ 9273.578697] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9273.593262] Lustre: 286953:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9273.677715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9280.633809] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9280.652101] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9280.795109] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9281.686890] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9292.470215] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9298.052635] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9308.026100] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9316.238725] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9357.794843] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9357.797633] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9357.819254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9357.819262] Lustre: Skipped 3 previous similar messages [ 9357.861783] Lustre: Skipped 1 previous similar message [ 9359.639279] Lustre: server umount lustre-OST0000 complete [ 9368.032525] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9368.088729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9368.113376] Lustre: Skipped 2 previous similar messages [ 9370.351723] Lustre: server umount lustre-MDT0000 complete [ 9373.152970] LustreError: 286128:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9373.184569] LustreError: 286128:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9374.601785] LustreError: 284975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841044 with bad export cookie 12504274400479831083 [ 9374.627044] LustreError: 284975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9374.629201] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9375.048714] Lustre: server umount lustre-MDT0001 complete [ 9383.424439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9387.480577] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9392.257858] Lustre: server umount MGS complete [ 9403.837982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9404.259722] LustreError: 290811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9404.286732] LustreError: 290824:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9404.346329] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9409.511182] LustreError: 290812:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9409.761548] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9418.605550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9424.232563] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9432.481881] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9438.621839] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9446.619878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9446.845102] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9446.855385] Lustre: Skipped 1 previous similar message [ 9454.550408] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9463.194587] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9495.011352] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9495.012510] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9495.026566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9495.043383] Lustre: Skipped 2 previous similar messages [ 9495.052804] Lustre: Skipped 1 previous similar message [ 9507.298054] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9507.553846] Lustre: server umount lustre-OST0000 complete [ 9511.393356] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9511.400750] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9511.428204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9511.433157] Lustre: Skipped 4 previous similar messages [ 9517.476854] Lustre: server umount lustre-MDT0000 complete [ 9520.608252] LustreError: 290811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9520.629049] LustreError: 290811:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9521.442395] LustreError: 290793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841191 with bad export cookie 12504274400479832924 [ 9521.456084] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9521.798773] Lustre: server umount lustre-MDT0001 complete [ 9533.358733] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 9536.282424] Key type lgssc unregistered [ 9536.572657] LNet: 294766:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9536.578346] LNetError: 294766:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9536.597156] LNet: Removed LNI 192.168.203.139@tcp [ 9537.564301] Key type .llcrypt unregistered [ 9537.571193] Key type ._llcrypt unregistered [ 9581.546358] Key type ._llcrypt registered [ 9581.550230] Key type .llcrypt registered [ 9581.690233] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 9598.368492] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 9599.597055] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9599.739526] alg: No test for adler32 (adler32-zlib) [ 9600.821695] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [ 9601.128879] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 9602.816374] Key type lgssc registered [ 9604.049569] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9645.062148] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9645.082381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9646.398395] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9646.453616] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9646.596676] Lustre: lustre-MDT0000: new disk, initializing [ 9646.728899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9646.746358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9651.125440] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9664.222118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9664.433687] Lustre: 298724:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9664.544355] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9664.560446] Lustre: Skipped 1 previous similar message [ 9664.726661] Lustre: lustre-MDT0001: new disk, initializing [ 9664.882056] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9664.965649] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9664.992995] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9672.450876] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9684.983696] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9692.187058] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9698.573111] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9698.929434] Lustre: lustre-OST0000: new disk, initializing [ 9698.933669] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9698.946360] Lustre: 300696:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9699.024886] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9705.320285] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9707.183711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9707.199130] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9707.345510] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9716.001673] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9724.089744] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9739.198345] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9748.416237] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9748.421585] Lustre: Skipped 1 previous similar message [ 9804.789150] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9804.807860] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9804.865944] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9804.874597] Lustre: Skipped 1 previous similar message [ 9807.540868] Lustre: server umount lustre-OST0000 complete [ 9813.495105] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9813.511972] LustreError: Skipped 1 previous similar message [ 9813.516201] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9813.531739] Lustre: Skipped 1 previous similar message [ 9813.536575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9815.015322] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9815.023359] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9818.884935] Lustre: server umount lustre-MDT0000 complete [ 9822.658833] LustreError: 298716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841492 with bad export cookie 7889470102512989082 [ 9822.659440] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9822.666544] LustreError: 298716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9823.198057] Lustre: server umount lustre-MDT0001 complete [ 9832.019916] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9836.867304] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9842.987302] Lustre: server umount MGS complete [ 9854.801176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9855.263698] LustreError: 304519:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9855.287224] LustreError: 304519:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9855.299821] LustreError: 304532:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9855.378441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9859.781529] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9860.584213] LustreError: 304520:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9865.700736] LustreError: 304519:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9867.945115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9868.276673] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9872.767502] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9879.529073] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9884.663398] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9890.684151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9891.078556] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9896.511683] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 9903.266275] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9936.352766] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9936.359624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9936.364835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9936.371352] Lustre: Skipped 1 previous similar message [ 9936.383199] Lustre: Skipped 2 previous similar messages [ 9941.474809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9941.489028] Lustre: Skipped 1 previous similar message [ 9949.151658] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9949.407626] Lustre: server umount lustre-OST0000 complete [ 9955.297797] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9955.310682] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9955.321886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9955.327889] Lustre: Skipped 2 previous similar messages [ 9960.277901] Lustre: server umount lustre-MDT0000 complete [ 9961.952743] LustreError: 305249:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9961.976148] LustreError: 305249:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9964.038952] LustreError: 305256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841633 with bad export cookie 7889470102512990944 [ 9964.041551] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9964.048051] LustreError: 305256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9964.437405] Lustre: server umount lustre-MDT0001 complete [ 9976.087148] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 9979.516287] Key type lgssc unregistered [ 9979.799659] LNet: 308471:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9979.806983] LNetError: 308471:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9979.828486] LNet: Removed LNI 192.168.203.139@tcp [ 9980.699459] Key type .llcrypt unregistered [ 9980.712937] Key type ._llcrypt unregistered [ 9999.758024] Key type ._llcrypt registered [ 9999.763500] Key type .llcrypt registered [ 9999.878456] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [10014.726292] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [10016.283560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10016.604877] alg: No test for adler32 (adler32-zlib) [10017.762621] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [10018.143898] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [10019.879282] Key type lgssc registered [10021.504274] Lustre: Echo OBD driver; http://www.lustre.org/ [10047.103139] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [10063.805431] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10063.854638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10065.189677] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10065.262705] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10065.339227] Lustre: lustre-MDT0000: new disk, initializing [10065.512417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10065.545756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10070.546826] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [10082.267076] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10082.496938] Lustre: lustre-OST0000: new disk, initializing [10082.501105] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10082.505471] Lustre: Skipped 1 previous similar message [10082.511982] Lustre: 312522:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10082.587251] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10090.354851] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [10090.624286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10090.636123] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10090.679087] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11083.748765] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11083.761338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11086.434933] Lustre: server umount lustre-MDT0000 complete [11089.934098] LustreError: 311709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842759 with bad export cookie 2401547229328859333 [11089.944990] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11090.084911] Lustre: server umount lustre-OST0000 complete [11113.373952] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [11125.160847] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [11160.015729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11160.253977] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11160.335184] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11160.441520] Lustre: lustre-MDT0000: new disk, initializing [11160.579517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11160.596439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11165.399847] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11179.560823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11179.811866] Lustre: 316887:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11179.968876] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11179.981714] Lustre: Skipped 1 previous similar message [11180.220476] Lustre: lustre-MDT0001: new disk, initializing [11180.406994] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11180.492157] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11180.511412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11185.495173] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11194.796630] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11200.725926] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11207.245241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11207.533745] Lustre: lustre-OST0000: new disk, initializing [11207.541349] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11207.546535] Lustre: 318867:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11207.608078] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11213.219816] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11217.988730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11218.000230] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11218.128473] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11221.716948] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11226.484468] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11226.676754] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11231.290135] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11231.516293] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11233.760176] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11233.766407] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11233.771442] LustreError: Skipped 1 previous similar message [11233.772511] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11233.809317] Lustre: Skipped 1 previous similar message [11238.882373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11238.886889] Lustre: Skipped 2 previous similar messages [11239.535966] Lustre: server umount lustre-OST0000 complete [11244.000435] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11244.006828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11244.027026] Lustre: Skipped 1 previous similar message [11244.045945] Lustre: Skipped 1 previous similar message [11249.120780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11249.131906] Lustre: Skipped 1 previous similar message [11258.335450] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11258.520957] Lustre: server umount lustre-MDT0000 complete [11262.320311] LustreError: 317681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842932 with bad export cookie 2401547229331800152 [11262.334215] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11262.339388] LustreError: 317681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11262.824382] Lustre: server umount lustre-MDT0001 complete [11266.670661] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11268.493634] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:02:17 (1787842937) [11275.547158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11276.120718] LustreError: 321182:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11276.140051] LustreError: 321182:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11276.268964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11281.019798] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11281.383705] LustreError: 321183:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11286.499036] LustreError: 321182:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11289.592761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11290.148582] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11294.728386] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11303.046923] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11309.381493] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11316.905406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11317.210460] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11322.085316] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11329.395811] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11336.356880] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11336.545867] Lustre: lustre-OST0001: new disk, initializing [11336.550587] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11336.559984] Lustre: 324018:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11336.633719] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11341.630501] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11345.024113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11345.035270] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11345.148875] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11346.219028] LustreError: 324539:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11346.232427] LustreError: 324539:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11347.235436] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11349.282296] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11349.293038] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11350.546799] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11350.764482] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11353.381649] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11353.401404] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11355.740752] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b87917c1800.ost_server_uuid 50 [11357.124168] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b87917c1800.ost_server_uuid in FULL state after 0 sec [11361.639862] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11361.658992] LustreError: 324529:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [11361.950501] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8b87917c1800.ost_server_uuid 50 [11363.421657] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8b87917c1800.ost_server_uuid in FULL state after 0 sec [11364.729030] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11380.192566] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11380.197976] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11380.208441] LustreError: Skipped 1 previous similar message [11380.210666] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11380.220321] Lustre: Skipped 1 previous similar message [11380.222789] Lustre: Skipped 2 previous similar messages [11384.084357] Lustre: server umount lustre-OST0001 complete [11386.352020] LustreError: 323004:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11386.369496] LustreError: 323004:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11391.463405] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11391.467642] LustreError: 323004:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11391.488844] Lustre: Skipped 1 previous similar message [11391.511941] LustreError: 323004:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11395.222979] Lustre: server umount lustre-OST0000 complete [11401.697955] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11401.719660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11401.726926] Lustre: Skipped 6 previous similar messages [11405.365444] Lustre: server umount lustre-MDT0000 complete [11406.817793] LustreError: 321182:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11406.845036] LustreError: 321182:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11409.253960] LustreError: 323026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843079 with bad export cookie 2401547229331801034 [11409.262618] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11409.268343] LustreError: 323026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11409.606853] Lustre: server umount lustre-MDT0001 complete [11420.219535] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [11422.926715] Key type lgssc unregistered [11423.237593] LNet: 326708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11423.246862] LNetError: 326708:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11423.269066] LNet: Removed LNI 192.168.203.139@tcp [11424.164151] Key type .llcrypt unregistered [11424.167909] Key type ._llcrypt unregistered [11447.694562] Key type ._llcrypt registered [11447.697486] Key type .llcrypt registered [11447.786184] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [11449.163101] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11449.211467] alg: No test for adler32 (adler32-zlib) [11450.248327] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [11450.556521] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [11452.231282] Key type lgssc registered [11453.491853] Lustre: Echo OBD driver; http://www.lustre.org/ [11461.495287] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11461.541255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11463.093643] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11467.814468] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11472.354858] LustreError: 328131:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11472.370640] LustreError: 328131:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11475.651766] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11477.474811] LustreError: 328130:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11482.594371] LustreError: 328131:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11482.714206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11483.156120] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11487.861366] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11497.466811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11497.799702] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11503.592252] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11508.022141] LustreError: 329808:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11508.038676] LustreError: 329808:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11508.054671] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11510.767946] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11516.391234] LustreError: 330393:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11516.402547] LustreError: 330393:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11516.674979] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11516.838292] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11521.198340] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11522.033792] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11527.259586] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11534.034754] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11534.247218] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11644.918200] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11644.926793] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11644.950899] Lustre: Skipped 1 previous similar message [11644.961295] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11644.973775] Lustre: Skipped 1 previous similar message [11649.200639] Lustre: server umount lustre-OST0001 complete [11650.035577] LustreError: 329810:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11650.057646] LustreError: 329810:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11655.142163] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11655.157393] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11655.165341] Lustre: Skipped 1 previous similar message [11660.265247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11660.272784] Lustre: Skipped 1 previous similar message [11661.401841] Lustre: server umount lustre-OST0000 complete [11665.381237] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11665.397082] Lustre: Skipped 1 previous similar message [11665.405465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11665.414207] Lustre: Skipped 1 previous similar message [11670.499274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11670.504659] Lustre: Skipped 1 previous similar message [11671.465995] Lustre: server umount lustre-MDT0000 complete [11675.474791] LustreError: 331813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843345 with bad export cookie 13926815949968491352 [11675.486446] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11675.489517] LustreError: 331813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11676.039662] Lustre: server umount lustre-MDT0001 complete [11689.337829] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [11692.233793] Key type lgssc unregistered [11692.564776] LNet: 333075:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11692.572864] LNetError: 333075:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11692.589915] LNet: Removed LNI 192.168.203.139@tcp [11693.446359] Key type .llcrypt unregistered [11693.448344] Key type ._llcrypt unregistered [11736.889802] Key type ._llcrypt registered [11736.899612] Key type .llcrypt registered [11737.042507] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [11737.991120] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11738.015224] alg: No test for adler32 (adler32-zlib) [11739.161467] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [11739.508891] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [11741.223263] Key type lgssc registered [11742.462143] Lustre: Echo OBD driver; http://www.lustre.org/ [11752.685206] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11752.709953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11754.212587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11759.006911] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11764.712688] LustreError: 335367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11764.727607] LustreError: 335367:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11768.983332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11769.517927] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11774.680093] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11777.795236] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11783.964794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11784.395281] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11790.748361] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11795.692555] LustreError: 336858:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11800.816198] LustreError: 336859:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11800.843809] LustreError: 336859:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11800.845903] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11801.314617] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11801.489709] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11806.731400] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11809.084405] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11818.140911] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11824.940630] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11837.412671] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11837.423308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11837.432243] Lustre: Skipped 2 previous similar messages [11837.436064] Lustre: Skipped 1 previous similar message [11842.536524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11842.547715] Lustre: Skipped 5 previous similar messages [11845.253504] Lustre: server umount lustre-MDT0000 complete [11847.649705] LustreError: 335362:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11847.666705] LustreError: 335362:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11848.762461] LustreError: 338378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843518 with bad export cookie 13257142906851725177 [11848.773238] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11848.775805] LustreError: 338378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11849.109503] Lustre: server umount lustre-MDT0001 complete [11863.143657] Lustre: server umount lustre-OST0000 complete [11877.073422] Lustre: server umount lustre-OST0001 complete [11895.422455] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [11907.411982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11907.797794] LustreError: 340634:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11907.912710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11913.416801] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11917.288938] LustreError: 340635:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11917.311404] LustreError: 340635:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11922.771024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11923.089349] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11927.884651] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11930.575789] Lustre: 341774:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11937.782747] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11944.496121] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11947.448598] LustreError: 342128:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11947.472426] LustreError: 342128:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11952.580836] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11953.206964] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11953.495991] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11953.505782] Lustre: Skipped 1 previous similar message [11958.777357] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11959.653421] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [11967.657208] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11971.558443] Lustre: 343645:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11979.267533] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11979.268939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11979.287965] Lustre: Skipped 4 previous similar messages [11979.297391] Lustre: Skipped 3 previous similar messages [11983.376226] Lustre: server umount lustre-MDT0000 complete [11984.354787] LustreError: 343254:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11984.376438] LustreError: 343254:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11987.180854] LustreError: 341776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843657 with bad export cookie 13257142906851726878 [11987.186977] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11987.192827] LustreError: 341776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11987.673973] Lustre: server umount lustre-MDT0001 complete [12001.335387] Lustre: server umount lustre-OST0000 complete [12005.344152] Lustre: 334629:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787843659/real 1787843659] req@ffff8eee28d07c00 x1874689690548992/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787843675 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12005.423365] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12006.271872] Lustre: server umount lustre-OST0001 complete [12010.967504] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12013.083506] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12015.373185] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:14:43 (1787843683) [12041.381826] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [12051.451067] loop0: detected capacity change from 0 to 1099511627776 [12052.008147] SGI XFS with ACLs, security attributes, quota, no debug enabled [12052.016231] *** VALIDATE xfs *** [12052.038288] XFS (loop0): Mounting V5 Filesystem [12052.053847] XFS (loop0): Ending clean mount [12053.231507] loop1: detected capacity change from 0 to 18691697672192 [12160.952672] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [12170.429859] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [12209.185493] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12209.435757] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12209.531486] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12209.644653] Lustre: lustre-MDT0000: new disk, initializing [12209.754439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12209.773677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12214.233389] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12225.033651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12225.161167] Lustre: 348781:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12225.237198] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12225.241590] Lustre: Skipped 1 previous similar message [12225.340803] Lustre: lustre-MDT0001: new disk, initializing [12225.552702] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12225.565510] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12231.308662] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12240.906674] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12246.890916] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12254.094716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12254.590823] Lustre: lustre-OST0000: new disk, initializing [12254.600755] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12254.608766] Lustre: 350747:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12256.082358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12256.091867] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12256.257737] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12261.422310] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12271.278328] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12275.262839] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12275.428380] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12279.544340] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12279.853565] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12281.842875] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12281.848749] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12281.848749] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12281.865250] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12281.914528] Lustre: Skipped 1 previous similar message [12286.950722] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12286.966347] Lustre: Skipped 1 previous similar message [12287.895505] Lustre: server umount lustre-OST0000 complete [12292.065580] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12292.066537] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12292.111834] Lustre: Skipped 2 previous similar messages [12297.185895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12297.193706] Lustre: Skipped 3 previous similar messages [12306.400976] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12306.640762] Lustre: server umount lustre-MDT0000 complete [12307.424727] LustreError: 348786:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12307.456128] LustreError: 348786:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12310.238763] LustreError: 350757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843980 with bad export cookie 13257142906851728432 [12310.240840] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12310.246567] LustreError: 350757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12310.594192] Lustre: server umount lustre-MDT0001 complete [12318.510175] XFS (loop0): Unmounting Filesystem [12322.471805] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:19:51 (1787843991) [12328.043475] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12328.501328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12328.509828] Lustre: Skipped 2 previous similar messages [12332.382764] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12339.854365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12344.263926] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12351.470820] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12357.146334] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12363.781896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12369.914560] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12376.170488] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12385.761590] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12385.770085] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12385.779048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12385.783939] Lustre: Skipped 2 previous similar messages [12388.833703] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12390.111982] Lustre: server umount lustre-OST0000 complete [12408.287140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12410.500915] Lustre: server umount lustre-MDT0000 complete [12413.913079] LustreError: 353287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844083 with bad export cookie 13257142906851729307 [12413.923732] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12413.931674] LustreError: 353287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12414.201461] Lustre: server umount lustre-MDT0001 complete [12423.731289] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [12426.035108] Key type lgssc unregistered [12426.321843] LNet: 356942:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12426.331024] LNetError: 356942:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12426.349373] LNet: Removed LNI 192.168.203.139@tcp [12427.032233] Key type .llcrypt unregistered [12427.034218] Key type ._llcrypt unregistered [12454.104560] Key type ._llcrypt registered [12454.107381] Key type .llcrypt registered [12454.249427] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [12455.288971] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12455.420516] alg: No test for adler32 (adler32-zlib) [12456.540510] Lustre: Lustre: Build Version: 2.17.57_60_g13a1c7b [12456.825332] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [12458.464147] Key type lgssc registered [12459.447675] Lustre: Echo OBD driver; http://www.lustre.org/ [12467.194602] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12467.216038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12468.719291] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12473.637125] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12478.949383] LustreError: 358510:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12478.956294] LustreError: 358510:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12482.268924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12482.602488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12487.784328] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12496.069490] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12503.784243] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12510.027528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12510.453448] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12516.360132] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12523.637131] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12535.941841] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.39@tcp (stopping) [12538.855588] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12538.866371] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12538.896817] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12539.376974] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12541.029249] Lustre: server umount lustre-MDT0001 complete [12541.031947] LustreError: 358507:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12544.489852] LustreError: 358511:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12549.608332] LustreError: 359238:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12549.625691] LustreError: 359238:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12553.380735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12553.466752] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12553.471225] Lustre: Found index 1 for lustre-MDT0001, updating log [12553.531119] Lustre: 358498:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12553.951609] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12559.190704] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12559.346800] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12560.384804] LustreError: 362029:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12560.395814] LustreError: 362029:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12560.417086] LustreError: 358502:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12560.462908] LustreError: 362035:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.139@tcp: cfg command failed: rc = -17 [12560.478036] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12560.478036] [12569.585868] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12569.591477] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12569.601663] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12569.614101] Lustre: Skipped 1 previous similar message [12569.635221] Lustre: Skipped 3 previous similar messages [12571.770851] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.39@tcp (stopping) [12574.595157] Lustre: server umount lustre-MDT0001 complete [12574.695653] LustreError: 359691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12574.709628] LustreError: 359691:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12584.935124] LustreError: 358511:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12584.955776] LustreError: 358511:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12585.607640] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12585.735305] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12585.751106] Lustre: Found index 1 for lustre-MDT0001, updating log [12585.817754] Lustre: 358498:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12586.115646] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12590.207915] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12591.593733] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12592.625529] LustreError: 363164:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12592.640093] LustreError: 363164:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12592.669788] LustreError: 358502:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12592.690514] LustreError: 358502:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12592.786970] LustreError: 363178:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12592.801173] LustreError: 363178:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12592.807704] LustreError: 363178:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.139@tcp: cfg command failed: rc = -17 [12592.817772] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.139@tcp [12592.817772] [12898.789525] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12898.793905] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12898.820055] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12898.828895] Lustre: Skipped 1 previous similar message [12901.572575] Lustre: server umount lustre-MDT0001 complete [12903.911756] LustreError: 359691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12903.928278] LustreError: 359691:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12911.300451] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12911.437689] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12911.445908] Lustre: Found index 1 for lustre-MDT0001, updating log [12911.476913] Lustre: 358500:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12911.892224] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12916.645290] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [12917.229032] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12920.820613] LustreError: 371908:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12920.829898] LustreError: 371908:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12920.848137] LustreError: 358502:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12920.861746] LustreError: 358502:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12920.946565] LustreError: 371911:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12920.993883] LustreError: 371911:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.139@tcp: cfg command failed: rc = -114 [12921.013063] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12921.013063] [13239.777732] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13239.780184] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13239.781024] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13239.781030] Lustre: Skipped 1 previous similar message [13239.794139] Lustre: Skipped 2 previous similar messages [13244.728934] Lustre: server umount lustre-MDT0000 complete [13244.900926] LustreError: 366623:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13244.929295] LustreError: 366623:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [13248.371837] LustreError: 362770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844918 with bad export cookie 4131307921798425543 [13248.399138] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13250.021253] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13250.042246] Lustre: Skipped 1 previous similar message [13254.830839] Lustre: server umount lustre-MDT0001 complete [13259.474690] Lustre: server umount lustre-OST0000 complete [13270.435266] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [13280.363472] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [13319.760363] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 11:36:28 (1787844988) [13326.980813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13327.203861] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13327.238380] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13327.320951] Lustre: lustre-MDT0000: new disk, initializing [13327.427641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13327.447457] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13332.497327] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [13343.747545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13343.889173] Lustre: 382741:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13344.010522] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13344.023334] Lustre: Skipped 1 previous similar message [13344.170421] Lustre: lustre-MDT0001: new disk, initializing [13344.261904] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13344.293161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13344.312803] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13348.290610] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [13359.961711] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13366.039968] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13372.960839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13373.194429] Lustre: lustre-OST0000: new disk, initializing [13373.198933] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13373.206352] Lustre: 384708:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13373.282447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13378.532971] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [13381.204623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13381.226482] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13381.346928] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13387.123423] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13395.425122] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13395.431098] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13395.443201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13395.451566] Lustre: Skipped 4 previous similar messages [13396.333810] Lustre: server umount lustre-MDT0000 complete [13396.963790] LustreError: 382751:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13396.978349] LustreError: 382751:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13399.577769] LustreError: 382731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845069 with bad export cookie 4131307921798427496 [13399.582257] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13399.589763] LustreError: 382731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [13399.869747] Lustre: server umount lustre-MDT0001 complete [13414.042175] Lustre: server umount lustre-OST0000 complete [13442.046503] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [13451.955278] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [13483.159291] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13195 sec ========= 11:39:11 (1787845151) [13484.922062] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 11:39:13 (1787845153) === [13488.428437] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 11:39:16 (1787845156) === [13523.716953] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [13525.994886] Key type lgssc unregistered [13526.231347] LNet: 391074:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13526.256686] LNetError: 391074:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13526.272549] LNet: Removed LNI 192.168.203.139@tcp [13527.052325] Key type .llcrypt unregistered [13527.053688] Key type ._llcrypt unregistered