[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 526748750 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002281] x2apic enabled [ 0.003007] Switched APIC routing to physical x2apic. [ 0.004009] kvm-guest: setup PV IPIs [ 0.006261] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008012] pid_max: default: 32768 minimum: 301 [ 0.009129] LSM: Security Framework initializing [ 0.010059] Yama: becoming mindful. [ 0.011033] SELinux: Initializing. [ 0.012073] *** VALIDATE selinux *** [ 0.020454] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024750] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025201] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027101] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028127] *** VALIDATE tmpfs *** [ 0.029476] *** VALIDATE proc *** [ 0.031099] *** VALIDATE cgroup *** [ 0.032011] *** VALIDATE cgroup2 *** [ 0.033297] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034168] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035014] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036027] Spectre V2 : User space: Vulnerable [ 0.037012] Speculative Store Bypass: Vulnerable [ 0.040123] debug: unmapping init [mem 0xffffffffb3259000-0xffffffffb3260fff] [ 0.042152] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043960] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045025] ... version: 2 [ 0.046012] ... bit width: 48 [ 0.047041] ... generic registers: 4 [ 0.048015] ... value mask: 0000ffffffffffff [ 0.049015] ... max period: 00007fffffffffff [ 0.050014] ... fixed-purpose events: 3 [ 0.051012] ... event mask: 000000070000000f [ 0.052285] rcu: Hierarchical SRCU implementation. [ 0.054329] smp: Bringing up secondary CPUs ... [ 0.055624] x86: Booting SMP configuration: [ 0.056024] .... node #0, CPUs: #1 #2 #3 [ 0.062618] smp: Brought up 1 node, 4 CPUs [ 0.064011] smpboot: Max logical packages: 1 [ 0.065013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.135504] node 0 deferred pages initialised in 67ms [ 0.140080] devtmpfs: initialized [ 0.141284] x86/mm: Memory block size: 128MB [ 0.145233] gcov: version magic: 0x41383552 [ 0.149297] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.153206] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156262] pinctrl core: initialized pinctrl subsystem [ 0.159626] [ 0.160011] ************************************************************* [ 0.165053] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168014] ** ** [ 0.172014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.175011] ** ** [ 0.178025] ** This means that this kernel is built to expose internal ** [ 0.182014] ** IOMMU data structures, which may compromise security on ** [ 0.186030] ** your system. ** [ 0.189019] ** ** [ 0.193012] ** If you see this message and you are not debugging the ** [ 0.196023] ** kernel, report this immediately to your vendor! ** [ 0.200015] ** ** [ 0.204013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.207011] ************************************************************* [ 0.210840] NET: Registered protocol family 16 [ 0.212439] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.216056] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.219074] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.222400] cpuidle: using governor menu [ 0.223919] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.225401] PCI: Using configuration type 1 for base access [ 0.228125] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.242159] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.244017] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.248072] cryptd: max_cpu_qlen set to 1000 [ 0.251272] ACPI: Added _OSI(Module Device) [ 0.253015] ACPI: Added _OSI(Processor Device) [ 0.254011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.256012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.261346] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.267588] ACPI: Interpreter enabled [ 0.268063] ACPI: PM: (supports S0 S3 S4 S5) [ 0.270013] ACPI: Using IOAPIC for interrupt routing [ 0.272099] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.277364] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.288554] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.291064] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.295027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.298111] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.304482] acpiphp: Slot [2] registered [ 0.306149] acpiphp: Slot [5] registered [ 0.307176] acpiphp: Slot [6] registered [ 0.309164] acpiphp: Slot [7] registered [ 0.311121] acpiphp: Slot [8] registered [ 0.312111] acpiphp: Slot [9] registered [ 0.315118] acpiphp: Slot [10] registered [ 0.317124] acpiphp: Slot [3] registered [ 0.318150] acpiphp: Slot [4] registered [ 0.320099] acpiphp: Slot [11] registered [ 0.321101] acpiphp: Slot [12] registered [ 0.323167] acpiphp: Slot [13] registered [ 0.324112] acpiphp: Slot [14] registered [ 0.326122] acpiphp: Slot [15] registered [ 0.328163] acpiphp: Slot [16] registered [ 0.329115] acpiphp: Slot [17] registered [ 0.331111] acpiphp: Slot [18] registered [ 0.333106] acpiphp: Slot [19] registered [ 0.335124] acpiphp: Slot [20] registered [ 0.337110] acpiphp: Slot [21] registered [ 0.338128] acpiphp: Slot [22] registered [ 0.340105] acpiphp: Slot [23] registered [ 0.343138] acpiphp: Slot [24] registered [ 0.344246] acpiphp: Slot [25] registered [ 0.347121] acpiphp: Slot [26] registered [ 0.349103] acpiphp: Slot [27] registered [ 0.351147] acpiphp: Slot [28] registered [ 0.356155] acpiphp: Slot [29] registered [ 0.357101] acpiphp: Slot [30] registered [ 0.359103] acpiphp: Slot [31] registered [ 0.360089] PCI host bridge to bus 0000:00 [ 0.362021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.364025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.366025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.369024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.372022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.374026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.376181] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.379115] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.382649] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.393523] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.398061] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.401020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.403019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.407018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.409632] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.412896] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.417053] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.423765] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.429013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.445025] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.451013] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.458249] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.466015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.472013] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.495014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.512961] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.536015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.546014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.563014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.572218] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.583014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.592016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.613016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.626000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.634015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.645020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.671014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.690619] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.701018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.711086] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.738017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.751076] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.763021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.772018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.802027] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.819254] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.821412] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.824368] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.827388] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.830245] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.836173] iommu: Default domain type: Passthrough [ 0.838442] SCSI subsystem initialized [ 0.840121] ACPI: bus type USB registered [ 0.841116] usbcore: registered new interface driver usbfs [ 0.843086] usbcore: registered new interface driver hub [ 0.845094] usbcore: registered new device driver usb [ 0.847155] pps_core: LinuxPPS API ver. 1 registered [ 0.848009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.851062] PTP clock support registered [ 0.854045] EDAC MC: Ver: 3.0.0 [ 0.856137] PCI: Using ACPI for IRQ routing [ 0.857981] NetLabel: Initializing [ 0.860011] NetLabel: domain hash size = 128 [ 0.861010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.863133] NetLabel: unlabeled traffic allowed by default [ 0.866131] vgaarb: loaded [ 0.868251] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.870015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.875770] clocksource: Switched to clocksource kvm-clock [ 1.001280] VFS: Disk quotas dquot_6.6.0 [ 1.004586] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.011258] *** VALIDATE ramfs *** [ 1.016942] *** VALIDATE hugetlbfs *** [ 1.020658] pnp: PnP ACPI init [ 1.025490] pnp: PnP ACPI: found 6 devices [ 1.076726] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.084147] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.093072] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.097283] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.101468] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.106603] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.111690] NET: Registered protocol family 2 [ 1.116356] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.121990] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.127378] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.135479] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.142177] TCP: Hash tables configured (established 65536 bind 65536) [ 1.148100] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.152442] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.156462] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.160778] NET: Registered protocol family 1 [ 1.166294] RPC: Registered named UNIX socket transport module. [ 1.170359] RPC: Registered udp transport module. [ 1.173384] RPC: Registered tcp transport module. [ 1.179266] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.183358] NET: Registered protocol family 44 [ 1.185013] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.187273] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.189282] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.191949] PCI: CLS 0 bytes, default 64 [ 1.194249] Unpacking initramfs... [ 2.751324] debug: unmapping init [mem 0xffff9267fcc54000-0xffff9267fffbffff] [ 2.756968] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.759643] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.763064] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.410325] Initialise system trusted keyrings [ 3.412236] Key type blacklist registered [ 3.414578] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.424594] zbud: loaded [ 3.427561] *** VALIDATE nfs *** [ 3.429047] *** VALIDATE nfs4 *** [ 3.430795] pstore: using deflate compression [ 3.434780] Platform Keyring initialized [ 3.556422] NET: Registered protocol family 38 [ 3.557981] Key type asymmetric registered [ 3.559881] Asymmetric key parser 'x509' registered [ 3.562126] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.566188] io scheduler mq-deadline registered [ 3.567513] io scheduler kyber registered [ 3.569181] io scheduler bfq registered [ 3.571185] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.574866] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.578248] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.581420] ACPI: Power Button [PWRF] [ 3.587492] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.595681] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.623204] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.636306] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.677465] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.705531] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.735801] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.740243] Non-volatile memory driver v1.3 [ 3.741964] Linux agpgart interface v0.103 [ 3.780906] virtio_blk virtio1: [vda] 146152 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.784324] vda: detected capacity change from 0 to 74829824 [ 3.817876] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.821802] vdb: detected capacity change from 0 to 1073741824 [ 3.840772] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.844172] vdc: detected capacity change from 0 to 2621440000 [ 3.865924] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.869610] vdd: detected capacity change from 0 to 2621440000 [ 3.891501] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.894850] vde: detected capacity change from 0 to 4294967296 [ 3.915329] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.918815] vdf: detected capacity change from 0 to 4294967296 [ 3.927687] libphy: Fixed MDIO Bus: probed [ 3.943181] usbcore: registered new interface driver usbserial_generic [ 3.945120] usbserial: USB Serial support registered for generic [ 3.948556] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.952586] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.954331] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.956640] mousedev: PS/2 mouse device common for all mice [ 3.961321] rtc_cmos 00:05: RTC can wake from S4 [ 3.966728] rtc_cmos 00:05: registered as rtc0 [ 3.968783] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.970696] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.975577] intel_pstate: CPU model not supported [ 3.982341] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.984833] hid: raw HID events driver (C) Jiri Kosina [ 3.988139] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.989021] usbcore: registered new interface driver usbhid [ 3.993807] usbhid: USB HID core driver [ 3.995793] drop_monitor: Initializing network drop monitor service [ 3.998663] Initializing XFRM netlink socket [ 4.000700] NET: Registered protocol family 10 [ 4.004324] Segment Routing with IPv6 [ 4.005910] NET: Registered protocol family 17 [ 4.007718] mpls_gso: MPLS GSO support [ 4.013460] RAS: Correctable Errors collector initialized. [ 4.015891] AVX version of gcm_enc/dec engaged. [ 4.017785] AES CTR mode by8 optimization enabled [ 4.103244] sched_clock: Marking stable (4103165762, 0)->(5019662763, -916497001) [ 4.106310] registered taskstats version 1 [ 4.108871] Loading compiled-in X.509 certificates [ 4.110922] zswap: loaded using pool lzo/zbud [ 4.134936] Key type big_key registered [ 4.146888] Key type encrypted registered [ 4.148527] ima: No TPM chip found, activating TPM-bypass! [ 4.150516] ima: Allocated hash algorithm: sha1 [ 4.152174] ima: No architecture policies found [ 4.153175] evm: Initialising EVM extended attributes: [ 4.154484] evm: security.selinux [ 4.155422] evm: security.ima [ 4.156423] evm: security.capability [ 4.157756] evm: HMAC attrs: 0x1 [ 4.160260] rtc_cmos 00:05: setting system clock to 2026-08-24 19:14:49 UTC (1787598889) [ 4.166103] debug: unmapping init [mem 0xffffffffb4203000-0xffffffffb43fffff] [ 4.169488] debug: unmapping init [mem 0xffffffffb2f82000-0xffffffffb3258fff] [ 4.177132] Write protecting the kernel read-only data: 28672k [ 4.180909] debug: unmapping init [mem 0xffffffffb1603000-0xffffffffb17fffff] [ 4.183988] debug: unmapping init [mem 0xffffffffb1f14000-0xffffffffb1ffffff] [ 4.229886] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.238826] systemd[1]: Detected virtualization kvm. [ 4.241786] systemd[1]: Detected architecture x86-64. [ 4.243845] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.271792] systemd[1]: No hostname configured. [ 4.273661] systemd[1]: Set hostname to . [ 4.275965] random: systemd: uninitialized urandom read (16 bytes read) [ 4.278565] systemd[1]: Initializing machine ID from random generator. [ 4.438649] random: systemd: uninitialized urandom read (16 bytes read) [ 4.441729] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.447104] random: systemd: uninitialized urandom read (16 bytes read) [ 4.450592] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 4.454881] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local File Systems. [ OK ] Reached target Swap. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.275471] device-mapper: uevent: version 1.0.3 [ 5.282183] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon.[ 6.107122] random: fast init done Starting dracut initqueue hook... [ 6.155511] virtio_net virtio0 ens2: renamed from eth0 [ 6.307844] scsi host0: ata_piix [ 6.328290] scsi host1: ata_piix [ 6.330621] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.334173] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.665329] random: crng init done [ 10.668119] random: 7 urandom warning(s) missed due to ratelimiting [ 10.859525] dracut-initqueue[588]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.534717] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.856355] printk: systemd: 26 output lines suppressed due to ratelimiting [ 13.157399] SELinux: Disabled at runtime. [ 13.215558] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 13.226156] systemd[1]: Detected virtualization kvm. [ 13.228654] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.808309] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.811987] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.819813] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.826222] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.830792] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.850266] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.858376] systemd[1]: Created slice system-getty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. Activating swap /dev/disk/by-label/SWAP... Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ 13.954062] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target rpc_pipefs.target. Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. Mounting Huge Pages File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Mounting Kernel Debug File System... Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 14.519760] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 15.016787] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 15.036173] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 15.167958] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 15.213164] EDAC sbridge: Ver: 1.1.2 [ 17.088714] Key type dns_resolver registered [ 17.425467] NFS: Registering the id_resolver key type [ 17.427250] Key type id_resolver registered [ 17.428905] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg616-server login: [ 67.023958] libcfs: loading out-of-tree module taints kernel. [ 67.196672] Key type ._llcrypt registered [ 67.199603] Key type .llcrypt registered [ 67.353224] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 89.445805] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 91.123188] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 91.133976] alg: No test for adler32 (adler32-zlib) [ 92.658889] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 93.949283] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 95.631155] Key type lgssc registered [ 97.463142] Lustre: Echo OBD driver; http://www.lustre.org/ [ 98.404257] hrtimer: interrupt took 3046840 ns [ 119.995081] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 166.413752] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 179.467307] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 179.532549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 180.892469] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 180.955929] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 181.092802] Lustre: lustre-MDT0000: new disk, initializing [ 181.197967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 181.224925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 185.396726] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 198.716021] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 198.812410] Lustre: 6510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 198.867397] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 198.876366] Lustre: Skipped 1 previous similar message [ 199.007819] Lustre: lustre-MDT0001: new disk, initializing [ 199.147183] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 199.183300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 199.195559] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 202.774638] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.325509] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 216.555986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 216.844560] Lustre: lustre-OST0000: new disk, initializing [ 216.854852] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 216.860429] Lustre: 8446:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 216.919045] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 219.252141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 219.260231] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 219.373513] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 222.486287] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 236.882838] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 237.057723] Lustre: lustre-OST0001: new disk, initializing [ 237.065474] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 237.074425] Lustre: 9518:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 237.171550] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 243.650527] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 246.893375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 246.923859] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 247.004102] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 255.288798] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 261.209279] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 267.393906] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing check_logdir /tmp/testlogs/ [ 272.175467] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing yml_node [ 276.914612] Lustre: DEBUG MARKER: Client: 2.17.57.80 [ 279.905698] Lustre: DEBUG MARKER: MDS: 2.17.57.80 [ 282.039676] Lustre: DEBUG MARKER: OSS: 2.17.57.80 [ 284.120803] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 24 15:19:27 EDT 2026 [ 301.124863] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 302.950651] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 318.945026] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 318.946560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 318.960874] Lustre: Skipped 3 previous similar messages [ 318.967414] Lustre: Skipped 3 previous similar messages [ 324.065282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 324.074544] Lustre: Skipped 3 previous similar messages [ 324.984479] Lustre: server umount lustre-MDT0000 complete [ 329.263958] LustreError: 6502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787599214 with bad export cookie 8834213154009206053 [ 329.273354] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 329.281912] LustreError: 6502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 334.310032] LustreError: 6517:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 334.311037] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 334.314629] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 334.335114] LustreError: 6517:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 334.365796] Lustre: Skipped 1 previous similar message [ 335.705131] Lustre: server umount lustre-MDT0001 complete [ 339.895702] Lustre: server umount lustre-OST0000 complete [ 344.291108] Lustre: server umount lustre-OST0001 complete [ 352.005712] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 362.107544] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 398.466503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 398.821694] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 398.852060] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 398.932896] Lustre: lustre-MDT0000: new disk, initializing [ 399.004473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 399.027981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 403.215860] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 413.049079] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 413.198469] Lustre: 15074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 413.262573] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 413.268353] Lustre: Skipped 1 previous similar message [ 413.409634] Lustre: lustre-MDT0001: new disk, initializing [ 413.545885] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 413.592748] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 413.608784] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 418.135814] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 427.700607] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 433.652844] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 439.936741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 440.230574] Lustre: lustre-OST0000: new disk, initializing [ 440.239856] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 440.251889] Lustre: 17046:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 440.384189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 441.430020] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 441.442796] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 441.569121] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 446.213437] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 454.869400] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 458.814338] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 459.048319] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 463.237522] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 463.416540] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 467.429311] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 467.435863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 467.446170] LustreError: Skipped 1 previous similar message [ 467.448571] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 467.457784] Lustre: Skipped 1 previous similar message [ 467.461200] Lustre: Skipped 1 previous similar message [ 471.497611] Lustre: server umount lustre-OST0000 complete [ 475.106720] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 475.118151] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 475.140410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 475.149610] Lustre: Skipped 1 previous similar message [ 481.359272] Lustre: server umount lustre-MDT0000 complete [ 482.783896] LustreError: 15922:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 482.805933] LustreError: 15922:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 484.578728] LustreError: 15067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787599369 with bad export cookie 8834213154009207586 [ 484.580564] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 484.590853] LustreError: 15067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 484.862950] Lustre: server umount lustre-MDT0001 complete [ 489.124224] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 15:22:52 (1787599372) [ 496.005860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 496.526439] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 496.584117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 500.382971] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 501.750305] LustreError: 19310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 505.829852] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 507.494160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 511.793214] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 518.206138] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 522.498676] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 528.483239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 534.952944] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 541.206073] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 544.600701] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 551.394065] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 551.405390] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 551.419942] Lustre: Skipped 2 previous similar messages [ 551.440786] Lustre: Skipped 4 previous similar messages [ 557.252326] Lustre: server umount lustre-MDT0000 complete [ 560.352386] LustreError: 19289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787599445 with bad export cookie 8834213154009208468 [ 560.357550] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 560.359873] LustreError: 19289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 560.688282] Lustre: server umount lustre-MDT0001 complete [ 574.075305] Lustre: server umount lustre-OST0000 complete [ 593.849523] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 602.677364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 603.151901] LustreError: 23688:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 603.224610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 603.231586] Lustre: Skipped 2 previous similar messages [ 607.673431] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 617.150682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 622.417663] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 625.420269] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 631.670761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 637.572376] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 645.540836] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 645.706957] Lustre: lustre-OST0001: new disk, initializing [ 645.712872] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 645.722805] Lustre: 26005:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 647.503785] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 647.521417] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 647.622596] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 651.872762] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 661.501928] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 665.066209] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 678.890658] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 678.900950] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 678.911096] Lustre: Skipped 1 previous similar message [ 678.917353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 678.926842] Lustre: Skipped 3 previous similar messages [ 683.426586] Lustre: server umount lustre-MDT0000 complete [ 684.518229] LustreError: 26015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 684.532714] LustreError: 26015:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 686.693712] LustreError: 23668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787599572 with bad export cookie 8834213154009209315 [ 686.695552] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 686.703910] LustreError: 23668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 687.211655] Lustre: server umount lustre-MDT0001 complete [ 701.688515] Lustre: server umount lustre-OST0000 complete [ 704.800246] Lustre: 3643:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787599574/real 1787599574] req@ffff92687fcbd500 x1874433387235712/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787599590 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 704.835035] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 704.854104] Lustre: Skipped 3 previous similar messages [ 705.257717] Lustre: server umount lustre-OST0001 complete [ 721.552298] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 731.675823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 732.243127] LustreError: 29023:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 732.337685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 732.360669] Lustre: Skipped 3 previous similar messages [ 736.768598] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 744.823186] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 749.438309] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 752.488301] Lustre: 30172:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 759.246667] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 765.892822] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 775.547855] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 782.643941] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 789.916852] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 793.403526] Lustre: 32050:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 801.762187] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 801.770893] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 801.780366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 801.784038] Lustre: Skipped 4 previous similar messages [ 805.658252] Lustre: server umount lustre-MDT0000 complete [ 807.909810] LustreError: 29020:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 807.923127] LustreError: 29020:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 809.032546] LustreError: 29003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787599694 with bad export cookie 8834213154009210603 [ 809.044270] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 809.057543] LustreError: 29003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 809.387722] Lustre: server umount lustre-MDT0001 complete [ 823.405928] Lustre: server umount lustre-OST0000 complete [ 835.971390] Lustre: server umount lustre-OST0001 complete [ 845.344572] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 15:28:49 (1787599729) [ 871.620968] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 881.726527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 887.474249] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 898.005255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 905.182400] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 909.095996] Lustre: 36001:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 917.440096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 924.447197] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 932.703657] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 938.480214] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 945.218862] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 948.364440] Lustre: 37880:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 953.315571] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 953.318869] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 953.338340] Lustre: Skipped 4 previous similar messages [ 953.353135] Lustre: Skipped 7 previous similar messages [ 959.527728] Lustre: server umount lustre-MDT0000 complete [ 962.922678] LustreError: 34833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787599848 with bad export cookie 8834213154009211716 [ 962.930040] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 962.942336] LustreError: 34833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 963.162851] Lustre: server umount lustre-MDT0001 complete [ 976.623757] Lustre: server umount lustre-OST0000 complete [ 979.935331] Lustre: 3643:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787599848/real 1787599848] req@ffff926871d09880 x1874433387333760/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787599864 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 980.323709] Lustre: server umount lustre-OST0001 complete [ 990.421742] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 15:31:14 (1787599874) [ 1004.680705] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1013.425369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1013.876956] LustreError: 40022:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1013.889702] LustreError: 40022:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 1013.957265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1013.963734] Lustre: Skipped 7 previous similar messages [ 1017.711655] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1025.849096] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1030.741866] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1033.541563] Lustre: 41171:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1040.010634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1046.248314] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1053.963341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1060.311562] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1067.316942] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1070.847987] Lustre: 43050:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1071.758749] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1071.763394] Lustre: Skipped 1 previous similar message [ 1080.793077] Lustre: server umount lustre-MDT0000 complete [ 1085.358568] LustreError: 41173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787599970 with bad export cookie 8834213154009212822 [ 1085.373986] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1085.378670] LustreError: 41173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1088.041084] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [ 1088.049674] Lustre: Skipped 13 previous similar messages [ 1090.034163] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1090.056947] Lustre: Skipped 8 previous similar messages [ 1091.846927] Lustre: server umount lustre-MDT0001 complete [ 1100.531619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1117.824962] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1121.270228] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1126.416366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1130.035060] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1130.045494] Lustre: Skipped 1 previous similar message [ 1131.617058] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1138.498578] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1139.781960] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1145.264340] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1146.771735] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1152.487853] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1157.313000] Lustre: server umount lustre-MDT0000 complete [ 1160.531253] LustreError: 40004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787600045 with bad export cookie 8834213154009213949 [ 1160.538488] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1160.543218] LustreError: 40004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1160.858701] Lustre: server umount lustre-MDT0001 complete [ 1166.846496] Lustre: server umount lustre-OST0000 complete [ 1170.313312] Lustre: server umount lustre-OST0001 complete [ 1179.085950] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 15:34:22 (1787600062) [ 1194.478754] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1204.811848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1209.439260] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1217.677107] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1222.315961] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1224.985163] Lustre: 48806:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1230.664830] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1237.665192] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1247.101991] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1254.409526] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1262.799289] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1267.153434] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1279.460315] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1285.600786] Lustre: server umount lustre-MDT0000 complete [ 1286.655843] LustreError: 47648:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1286.690525] LustreError: 47648:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 28 previous similar messages [ 1289.310212] LustreError: 47634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787600174 with bad export cookie 8834213154009214747 [ 1289.324255] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1289.334319] LustreError: 47634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1289.767862] Lustre: server umount lustre-MDT0001 complete [ 1304.449068] Lustre: server umount lustre-OST0000 complete [ 1318.320579] Lustre: server umount lustre-OST0001 complete [ 1328.255340] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 15:36:51 (1787600211) [ 1330.595047] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1332.414467] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 15:36:56 (1787600216) [ 1342.723373] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 1350.612833] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1371.708923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1371.991572] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1372.087792] Lustre: lustre-MDT0000: new disk, initializing [ 1372.191764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1377.060607] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1388.378222] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1394.965743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1395.331270] Lustre: lustre-OST0000: new disk, initializing [ 1395.335522] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1395.340452] Lustre: Skipped 1 previous similar message [ 1395.345844] Lustre: 54883:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1396.389647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1396.399473] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1396.435077] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1401.439213] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1410.657623] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1412.897250] Lustre: DEBUG MARKER: create test files [ 1447.927175] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1447.930383] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1447.968617] Lustre: Skipped 10 previous similar messages [ 1447.969473] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1447.969477] Lustre: Skipped 12 previous similar messages [ 1452.319588] Lustre: server umount lustre-OST0000 complete [ 1457.120580] Lustre: server umount lustre-MDT0000 complete [ 1469.052402] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 1472.243824] Key type lgssc unregistered [ 1472.553548] LNet: 56515:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1472.565875] LNetError: 56515:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1472.604272] LNet: Removed LNI 192.168.206.116@tcp [ 1473.460271] Key type .llcrypt unregistered [ 1473.462513] Key type ._llcrypt unregistered [ 1499.224983] Key type ._llcrypt registered [ 1499.227545] Key type .llcrypt registered [ 1499.327412] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1500.555858] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1500.608638] alg: No test for adler32 (adler32-zlib) [ 1501.634560] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 1501.923052] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 1503.591159] Key type lgssc registered [ 1504.577360] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1511.218891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1511.252673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1512.676037] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1517.130834] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1523.165046] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1528.829646] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1529.131611] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1534.698331] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1537.330300] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1537.464562] Lustre: 59240:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1537.474785] Lustre: 59240:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1537.482920] Lustre: 59240:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1537.493436] Lustre: 59240:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1537.503943] Lustre: 59240:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1537.512407] Lustre: 59240:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1541.290588] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1546.317863] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1551.144768] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1560.349510] Lustre: server umount lustre-OST0000 complete [ 1574.007696] Lustre: server umount lustre-MDT0000 complete [ 1584.215861] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 1586.641569] Key type lgssc unregistered [ 1586.983314] LNet: 60851:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1587.003231] LNetError: 60851:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1587.024207] LNet: Removed LNI 192.168.206.116@tcp [ 1588.003114] Key type .llcrypt unregistered [ 1588.005246] Key type ._llcrypt unregistered [ 1617.213043] Key type ._llcrypt registered [ 1617.217706] Key type .llcrypt registered [ 1617.332991] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1618.658509] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1618.693281] alg: No test for adler32 (adler32-zlib) [ 1619.813203] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 1620.054818] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 1621.695420] Key type lgssc registered [ 1622.874439] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1630.276180] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1630.309832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1631.854637] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1636.295685] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1644.195235] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1651.272190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1651.655955] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1656.780583] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1656.858712] Lustre: 63691:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1656.868836] Lustre: 63691:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1656.876445] Lustre: 63691:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1656.883924] Lustre: 63691:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1656.891891] Lustre: 63691:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1656.897398] Lustre: 63691:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1658.710585] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1665.991236] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1671.927517] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1682.408330] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1682.414771] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1682.431471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1686.163329] Lustre: server umount lustre-OST0000 complete [ 1690.107249] Lustre: server umount lustre-MDT0000 complete [ 1701.346991] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 1704.671670] Key type lgssc unregistered [ 1704.976629] LNet: 65252:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1704.987596] LNetError: 65252:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1705.006791] LNet: Removed LNI 192.168.206.116@tcp [ 1706.113906] Key type .llcrypt unregistered [ 1706.115710] Key type ._llcrypt unregistered [ 1724.735832] Key type ._llcrypt registered [ 1724.738378] Key type .llcrypt registered [ 1724.828589] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 1740.862965] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1742.172704] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1742.245353] alg: No test for adler32 (adler32-zlib) [ 1743.428779] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 1743.713520] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 1745.383333] Key type lgssc registered [ 1746.697807] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1782.678825] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1782.705470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1783.933343] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1783.958744] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1784.015969] Lustre: lustre-MDT0000: new disk, initializing [ 1784.077913] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1784.094851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1788.179875] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1798.774475] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1798.853954] Lustre: 68416:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1798.901374] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1798.907288] Lustre: Skipped 1 previous similar message [ 1798.997656] Lustre: lustre-MDT0001: new disk, initializing [ 1799.111513] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1799.142116] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1799.160735] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1803.506437] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1812.119387] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1817.500816] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1824.317986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1824.715318] Lustre: lustre-OST0000: new disk, initializing [ 1824.723979] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1824.732860] Lustre: 70383:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1824.870883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1830.701809] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1831.484384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1831.492257] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1831.582762] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1839.599915] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1844.456463] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1844.700569] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1849.076426] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1849.318414] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1852.391561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1852.394625] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1852.396300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1852.408838] Lustre: Skipped 1 previous similar message [ 1852.431139] LustreError: Skipped 1 previous similar message [ 1857.416909] Lustre: server umount lustre-OST0000 complete [ 1862.629782] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1862.640478] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1862.644313] Lustre: Skipped 1 previous similar message [ 1862.653351] Lustre: Skipped 2 previous similar messages [ 1867.624761] Lustre: server umount lustre-MDT0000 complete [ 1871.366877] LustreError: 68409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787600756 with bad export cookie 7501890455234689866 [ 1871.385414] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1871.705482] Lustre: server umount lustre-MDT0001 complete [ 1880.888768] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 15:46:04 (1787600764) [ 1893.209553] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1931.135095] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 1940.552952] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 1978.195722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1978.588948] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1978.630339] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1978.765713] Lustre: lustre-MDT0000: new disk, initializing [ 1978.892979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1978.919446] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1983.427740] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 1994.241840] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1994.449593] Lustre: 76404:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1994.560169] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1994.567317] Lustre: Skipped 1 previous similar message [ 1994.712295] Lustre: lustre-MDT0001: new disk, initializing [ 1994.955391] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1995.050487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1995.077183] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2001.201766] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2013.149728] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2019.939672] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2027.382453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2027.640256] Lustre: lustre-OST0000: new disk, initializing [ 2027.650149] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2027.661202] Lustre: 78375:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2027.763483] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2033.840342] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2037.828828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2037.838227] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2037.992040] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2042.745490] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2047.043401] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2047.269077] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2051.594845] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2051.804756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2053.604402] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2053.615301] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2053.619882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2053.628650] Lustre: Skipped 1 previous similar message [ 2053.645031] Lustre: Skipped 1 previous similar message [ 2058.725221] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2058.731030] Lustre: Skipped 1 previous similar message [ 2059.733878] Lustre: server umount lustre-OST0000 complete [ 2063.848764] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2063.865583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2063.873522] Lustre: Skipped 1 previous similar message [ 2069.959560] Lustre: server umount lustre-MDT0000 complete [ 2073.645646] LustreError: 78374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787600958 with bad export cookie 7501890455234690748 [ 2073.655132] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2073.661685] LustreError: 78374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2074.041435] Lustre: server umount lustre-MDT0001 complete [ 2082.626267] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 15:49:26 (1787600966) [ 2089.207086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2089.669032] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2089.690299] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2089.769389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2093.796377] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2095.083872] LustreError: 80789:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2100.195193] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2102.916481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2108.287450] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2116.357090] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2122.082607] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2130.061885] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2130.532021] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2130.537896] Lustre: Skipped 1 previous similar message [ 2137.348993] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2144.779929] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2147.847299] Lustre: 83459:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 6f26f75e-978e-46af-b600-2f06e653752a at adminstrative request [ 2147.941967] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2147.952962] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2158.568879] Lustre: MGS: Client 6f26f75e-978e-46af-b600-2f06e653752a (at 0@lo) reconnecting [ 2158.584157] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x681c119f5028422e to 0x681c119f502844ce [ 2158.597641] Lustre: MGC192.168.206.116@tcp: Connection restored to 0@lo (at 0@lo) [ 2185.541696] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2185.819944] Lustre: lustre-OST0001: new disk, initializing [ 2185.827343] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2185.837769] Lustre: 83854:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2192.787055] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2195.034780] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2195.047866] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2195.187131] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2202.152638] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2205.665581] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2205.677868] LustreError: Skipped 1 previous similar message [ 2205.678018] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2205.693602] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2205.699586] Lustre: Skipped 2 previous similar messages [ 2205.714744] Lustre: Skipped 3 previous similar messages [ 2211.299614] Lustre: server umount lustre-OST0001 complete [ 2215.907633] LustreError: 82612:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2215.908924] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2215.944360] LustreError: 82612:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2221.043498] LustreError: 82612:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2221.074316] LustreError: 82612:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2221.948177] Lustre: server umount lustre-OST0000 complete [ 2231.265139] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2231.277219] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2231.311095] Lustre: Skipped 7 previous similar messages [ 2233.753492] Lustre: server umount lustre-MDT0000 complete [ 2236.406055] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2236.453696] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2238.953068] LustreError: 81525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787601124 with bad export cookie 7501890455234692302 [ 2238.958482] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2238.963294] LustreError: 81525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2239.556327] Lustre: server umount lustre-MDT0001 complete [ 2249.412740] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 15:52:13 (1787601133) [ 2250.969854] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2253.216419] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 15:52:16 (1787601136) [ 2255.070523] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2256.981476] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 15:52:20 (1787601140) [ 2259.138356] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2260.838295] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 15:52:24 (1787601144) [ 2262.483336] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2264.187319] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 15:52:27 (1787601147) [ 2266.234945] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2271.953379] loop: module loaded [ 2271.976172] loop0: detected capacity change from 0 to 4294967296 [ 2272.032451] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2273.552997] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2276.904776] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2280.819264] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2313.368887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2313.422137] LustreError: 87978:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2313.430598] LustreError: 87978:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2313.440128] LustreError: 87978:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2313.463542] LustreError: 87978:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2334.146548] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 2344.250731] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 2378.884777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2379.110712] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2379.151592] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2379.257537] Lustre: lustre-MDT0000: new disk, initializing [ 2379.361806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2379.371826] Lustre: Skipped 1 previous similar message [ 2379.408208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2384.067525] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2394.360029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2394.493743] Lustre: 90847:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2394.775723] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2400.184946] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2409.967639] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2415.118345] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2422.638802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2422.912473] Lustre: 92812:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2428.795980] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2431.591816] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2431.743798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2437.122522] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2441.118399] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2441.278628] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2445.379462] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2445.560883] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2452.450807] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2452.452055] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2452.463055] Lustre: Skipped 3 previous similar messages [ 2452.469441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2452.482274] Lustre: Skipped 1 previous similar message [ 2453.573961] Lustre: server umount lustre-OST0000 complete [ 2471.391382] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2471.647718] Lustre: server umount lustre-MDT0000 complete [ 2471.909197] LustreError: 92819:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2471.931846] LustreError: 92819:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2475.867313] LustreError: 90839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787601361 with bad export cookie 7501890455234692918 [ 2475.877817] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2475.884532] LustreError: 90839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2476.166633] Lustre: server umount lustre-MDT0001 complete [ 2485.207702] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 15:56:08 (1787601368) [ 2491.410496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2496.294162] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2504.303182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2508.950989] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2516.313696] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2521.815280] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2528.678794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2528.957582] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2528.968138] Lustre: Skipped 4 previous similar messages [ 2534.377995] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2541.331341] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2545.839108] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2546.030185] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2550.433740] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2550.661921] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2557.662163] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2557.818957] Lustre: lustre-OST0001: new disk, initializing [ 2557.821686] Lustre: Skipped 2 previous similar messages [ 2557.824297] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2557.827141] Lustre: Skipped 3 previous similar messages [ 2557.830755] Lustre: 98596:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2564.653401] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2565.748536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2565.757696] Lustre: Skipped 2 previous similar messages [ 2565.764888] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2565.920055] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2573.234479] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2577.598912] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2577.774231] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2581.812286] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2581.979831] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2599.325251] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2620.547781] Lustre: Failing over lustre-MDT0000 [ 2620.966321] Lustre: server umount lustre-MDT0000 complete [ 2622.433756] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2622.438660] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2622.448100] LustreError: Skipped 1 previous similar message [ 2622.448863] LustreError: 95226:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2622.448871] LustreError: 95226:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2622.496932] Lustre: Skipped 5 previous similar messages [ 2637.795641] Lustre: 66640:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787601507/real 1787601507] req@000000008d34a925 x1874435117417344/t0(0) o400->MGC192.168.206.116@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787601523 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2637.837677] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2645.435190] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2645.438538] LDISKFS-fs (dm-0): recovery complete [ 2645.453816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2648.032931] LustreError: 66637:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000c6252ada x1874435117424256/t0(0) o250->MGC192.168.206.116@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2648.366748] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2651.666975] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2651.673902] Lustre: 96267:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2652.890297] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2653.674889] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2653.680880] Lustre: 96268:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2653.701601] Lustre: 96268:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2653.825122] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2654.151994] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2654.485900] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2654.501705] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2654.827811] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2654.835550] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2654.850685] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2654.874234] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2655.520196] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2655.532904] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2655.855305] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2655.863377] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2657.146128] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2657.164104] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2657.810478] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2657.831929] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2658.151650] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2658.158553] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2661.345184] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2661.364259] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2662.024243] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2662.042452] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2662.363036] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2662.373110] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2669.428718] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2669.438725] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2670.080394] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2670.093249] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2670.423108] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2670.430321] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2685.713493] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2685.729359] Lustre: 101441:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2686.376441] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2686.391308] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2686.728073] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2686.740408] LustreError: 101441:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2711.207665] Lustre: 101441:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2711.223326] Lustre: 101441:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2711.237416] Lustre: 101441:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 33701671-b25d-4ed6-8499-4c5bfb1e71d5@192.168.206.16@tcp [ 2711.259892] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2711.271610] Lustre: 101441:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000da9ec959 x1874435099398016/t0(8589934776) o35->33701671-b25d-4ed6-8499-4c5bfb1e71d5@192.168.206.16@tcp:187/0 lens 392/0 e 0 to 0 dl 1787601607 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2711.299545] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2711.315589] Lustre: lustre-MDT0000: Denying connection for new client 33701671-b25d-4ed6-8499-4c5bfb1e71d5 (at 192.168.206.16@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2711.334033] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2711.385740] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2711.404303] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2711.405669] Lustre: Skipped 2 previous similar messages [ 2711.449862] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2711.455114] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2724.842739] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2724.852157] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2724.878674] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2724.885786] Lustre: Skipped 6 previous similar messages [ 2728.764870] Lustre: server umount lustre-OST0000 complete [ 2730.491561] LustreError: 101816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2730.516299] LustreError: 101816:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 36 previous similar messages [ 2747.359127] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2747.668606] Lustre: server umount lustre-OST0001 complete [ 2757.869313] Lustre: server umount lustre-MDT0000 complete [ 2761.244893] LustreError: 95206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787601646 with bad export cookie 7501890455234744130 [ 2761.251879] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2761.261076] LustreError: 95206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2761.658935] Lustre: server umount lustre-MDT0001 complete [ 2769.209829] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:00:53 (1787601653) [ 2776.973486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2776.984554] Lustre: *** cfs_fail_loc=197, val=0*** [ 2792.610238] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2797.727850] LustreError: 103428:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.116@tcp: failed processing log, type 4: rc = -110 [ 2802.881466] Lustre: *** cfs_fail_loc=197, val=0*** [ 2802.883801] Lustre: Skipped 65 previous similar messages [ 2823.393377] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2823.405195] Lustre: Skipped 2 previous similar messages [ 2828.413800] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2834.784146] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2836.887505] Lustre: server umount lustre-OST0000 complete [ 2845.548553] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:02:09 (1787601729) [ 2865.325549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2865.846850] LustreError: 105048:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2865.867715] LustreError: 105048:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 2870.304826] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2878.826915] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2882.553405] LustreError: 105080:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2882.565026] LustreError: 105080:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2882.578994] LustreError: 105080:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 2882.836866] Lustre: server umount lustre-MDT0000 complete [ 2915.103584] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 2924.947546] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 2963.340091] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2963.470765] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2963.490394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2963.567301] Lustre: lustre-MDT0000: new disk, initializing [ 2963.639178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2968.157172] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2980.158674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2980.263566] Lustre: 109251:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2980.614884] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2984.898464] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 2992.929659] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2997.731585] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3003.315074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3003.560653] Lustre: 111220:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3009.150631] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3012.287203] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3018.055993] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3021.899991] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3022.051837] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3025.852345] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3026.081784] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3027.953542] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3027.965118] LustreError: Skipped 2 previous similar messages [ 3027.968428] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3027.978657] Lustre: Skipped 5 previous similar messages [ 3027.982386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3027.989926] Lustre: Skipped 11 previous similar messages [ 3042.272552] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3042.450806] Lustre: server umount lustre-OST0000 complete [ 3051.916441] Lustre: server umount lustre-MDT0000 complete [ 3054.930783] LustreError: 109244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787601940 with bad export cookie 7501890455234754791 [ 3054.931317] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3054.938770] LustreError: 109244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3055.186486] Lustre: server umount lustre-MDT0001 complete [ 3062.453072] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:05:46 (1787601946) [ 3070.889996] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3073.369775] Key type lgssc unregistered [ 3073.628878] LNet: 113714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3073.638974] LNetError: 113714:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3073.661125] LNet: Removed LNI 192.168.206.116@tcp [ 3074.300587] Key type .llcrypt unregistered [ 3074.304111] Key type ._llcrypt unregistered [ 3090.392164] Key type ._llcrypt registered [ 3090.394900] Key type .llcrypt registered [ 3090.514692] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 3104.439713] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3105.251897] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3105.330418] alg: No test for adler32 (adler32-zlib) [ 3106.301631] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 3106.485234] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3108.191226] Key type lgssc registered [ 3109.138585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3144.276085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3144.318029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3145.602090] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3145.640903] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3145.715980] Lustre: lustre-MDT0000: new disk, initializing [ 3145.776400] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3145.791840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3150.092936] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3159.621634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3159.823389] Lustre: lustre-OST0000: new disk, initializing [ 3159.826714] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3159.830783] Lustre: Skipped 1 previous similar message [ 3159.834811] Lustre: 117886:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3159.891868] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3160.450859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3160.459216] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3160.484067] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3165.426576] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3174.190027] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3174.362109] Lustre: lustre-OST0001: new disk, initializing [ 3174.364816] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3174.370972] Lustre: 118753:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3174.447916] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3179.729724] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3184.740216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3184.750541] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3184.789153] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3189.731556] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3189.736507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3189.745020] Lustre: Skipped 1 previous similar message [ 3189.761672] Lustre: Skipped 1 previous similar message [ 3193.772985] Lustre: server umount lustre-MDT0000 complete [ 3198.924890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3202.803392] LustreError: 117073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787602088 with bad export cookie 8075838109136601511 [ 3202.811517] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3202.860765] Lustre: server umount lustre-OST0000 complete [ 3205.726273] Lustre: server umount lustre-OST0001 complete [ 3212.369344] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 16:08:16 (1787602096) [ 3230.477463] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 16:08:34 (1787602114) [ 3239.327470] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3257.835900] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 16:09:01 (1787602141) [ 3293.511369] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 3303.299722] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3339.114106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3339.315525] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3339.332805] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3339.404817] Lustre: lustre-MDT0000: new disk, initializing [ 3339.494307] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3339.514054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3343.523424] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3352.147973] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3352.254693] Lustre: 126515:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3352.302613] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3352.307874] Lustre: Skipped 1 previous similar message [ 3352.407446] Lustre: lustre-MDT0001: new disk, initializing [ 3352.550537] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3352.591099] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3352.605321] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3356.828191] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3365.939700] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3371.819265] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3378.246531] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3378.538368] Lustre: lustre-OST0000: new disk, initializing [ 3378.541371] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3378.550363] Lustre: 128477:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3378.621610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3383.841625] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3387.448504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3387.465141] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3387.591609] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3391.870845] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3395.801551] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3395.949136] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3399.585946] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3399.780525] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3403.236117] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3403.240573] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3403.242077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3403.253764] Lustre: Skipped 1 previous similar message [ 3403.265935] LustreError: Skipped 1 previous similar message [ 3407.524746] Lustre: server umount lustre-OST0000 complete [ 3413.474029] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3413.477445] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3413.487253] Lustre: Skipped 1 previous similar message [ 3413.492411] Lustre: Skipped 2 previous similar messages [ 3418.592885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3418.607386] Lustre: Skipped 1 previous similar message [ 3422.690606] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3422.698027] Lustre: Skipped 1 previous similar message [ 3425.248618] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3425.472297] Lustre: server umount lustre-MDT0000 complete [ 3428.486829] LustreError: 126507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787602313 with bad export cookie 8075838109136602729 [ 3428.496767] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3428.500343] LustreError: 126507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3428.803939] Lustre: server umount lustre-MDT0001 complete [ 3436.819497] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 16:12:00 (1787602320) [ 3442.794204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3443.295404] LustreError: 130891:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3443.314518] LustreError: 130891:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3443.379684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3446.981507] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3448.800103] LustreError: 130892:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3452.898994] LustreError: 130891:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3454.236820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3458.348923] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3465.783111] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3470.997164] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3476.798130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3477.193344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3477.200179] Lustre: Skipped 1 previous similar message [ 3483.907624] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3492.044041] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3496.350846] Lustre: *** cfs_fail_loc=159, val=0*** [ 3512.915604] Lustre: lustre-MDT0000: Client 9ef78dd8-8041-4e39-a602-5382a11f7ebe (at 192.168.206.16@tcp) reconnecting [ 3514.167765] Lustre: *** cfs_fail_loc=159, val=0*** [ 3514.175262] Lustre: Skipped 5 previous similar messages [ 3530.834328] Lustre: lustre-MDT0000: Client 9ef78dd8-8041-4e39-a602-5382a11f7ebe (at 192.168.206.16@tcp) reconnecting [ 3535.843914] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3535.857238] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3535.880851] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3537.388468] Lustre: server umount lustre-OST0000 complete [ 3541.473467] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3541.480152] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3550.248858] Lustre: server umount lustre-MDT0000 complete [ 3553.718300] LustreError: 130872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787602439 with bad export cookie 8075838109136603611 [ 3553.719385] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3553.729406] LustreError: 130872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3554.201054] Lustre: server umount lustre-MDT0001 complete [ 3564.614981] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3567.070414] Key type lgssc unregistered [ 3567.342400] LNet: 134673:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3567.350542] LNetError: 134673:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3567.362392] LNet: Removed LNI 192.168.206.116@tcp [ 3568.137835] Key type .llcrypt unregistered [ 3568.141104] Key type ._llcrypt unregistered [ 3587.930868] Key type ._llcrypt registered [ 3587.933851] Key type .llcrypt registered [ 3588.034083] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3588.862118] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3588.971845] alg: No test for adler32 (adler32-zlib) [ 3590.005887] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 3590.238380] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3591.943482] Key type lgssc registered [ 3593.015819] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3600.038756] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3600.069964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3601.583308] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3605.607350] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3611.111110] LustreError: 136069:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3611.148661] LustreError: 136069:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3613.442655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3613.873095] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3618.200569] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3624.816772] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3630.681426] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3636.626470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3636.902801] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3642.740100] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3646.130289] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3649.485892] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3658.611074] Lustre: *** cfs_fail_loc=159, val=0*** [ 3674.167025] Lustre: lustre-MDT0000: Client 53382fe7-5ded-48eb-af88-1554582d08fa (at 192.168.206.16@tcp) reconnecting [ 3676.979524] Lustre: *** cfs_fail_loc=159, val=0*** [ 3693.661903] Lustre: lustre-MDT0001: Client 53382fe7-5ded-48eb-af88-1554582d08fa (at 192.168.206.16@tcp) reconnecting [ 3695.255706] Lustre: *** cfs_fail_loc=159, val=0*** [ 3695.257970] Lustre: Skipped 3 previous similar messages [ 3711.577418] Lustre: lustre-MDT0001: Client 53382fe7-5ded-48eb-af88-1554582d08fa (at 192.168.206.16@tcp) reconnecting [ 3716.170842] Lustre: 139100:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3718.537524] Lustre: *** cfs_fail_loc=159, val=0*** [ 3718.540982] Lustre: Skipped 4 previous similar messages [ 3734.114781] Lustre: lustre-MDT0000: Client 37fe388e-7e52-4995-83dd-e73b4a5759b2 (at 192.168.206.16@tcp) reconnecting [ 3735.719609] Lustre: *** cfs_fail_loc=159, val=0*** [ 3735.727269] Lustre: Skipped 14 previous similar messages [ 3751.040386] Lustre: lustre-MDT0000: Client 37fe388e-7e52-4995-83dd-e73b4a5759b2 (at 192.168.206.16@tcp) reconnecting [ 3752.753122] Lustre: 139392:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3759.074560] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3759.077927] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3759.091833] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3759.148602] Lustre: Skipped 1 previous similar message [ 3764.196327] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3764.205783] Lustre: Skipped 2 previous similar messages [ 3764.831589] Lustre: server umount lustre-OST0000 complete [ 3769.323125] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3769.331755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3769.336240] Lustre: Skipped 1 previous similar message [ 3771.370869] Lustre: server umount lustre-MDT0000 complete [ 3774.436543] LustreError: 136064:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3774.453340] LustreError: 136064:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3775.596670] LustreError: 136048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787602660 with bad export cookie 18375567184843810662 [ 3775.606082] LustreError: 136048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3775.607947] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3776.046499] Lustre: server umount lustre-MDT0001 complete [ 3785.175562] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3787.634385] Key type lgssc unregistered [ 3787.917168] LNet: 140446:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3787.930547] LNetError: 140446:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3787.951811] LNet: Removed LNI 192.168.206.116@tcp [ 3788.682287] Key type .llcrypt unregistered [ 3788.686210] Key type ._llcrypt unregistered [ 3809.314453] Key type ._llcrypt registered [ 3809.316423] Key type .llcrypt registered [ 3809.406860] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3810.268969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3810.340043] alg: No test for adler32 (adler32-zlib) [ 3811.386841] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 3811.608890] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3813.248377] Key type lgssc registered [ 3814.253757] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3821.271557] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3821.298613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3822.626190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3826.594325] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3831.778144] LustreError: 141861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3831.788185] LustreError: 141861:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3833.829757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3834.154491] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3838.311139] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3844.310502] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3848.478555] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3854.148577] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3854.428518] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3859.577214] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3859.590565] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3866.740085] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3867.175741] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3872.882474] Lustre: 144442:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3875.521340] Lustre: 144490:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3877.048254] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3878.042568] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3882.164536] Lustre: 144637:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3883.546681] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3884.654434] Lustre: 144736:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3890.668541] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3890.678828] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3890.692335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3892.713361] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3892.713833] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3892.733439] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3892.763457] Lustre: Skipped 1 previous similar message [ 3895.137789] Lustre: server umount lustre-OST0000 complete [ 3900.907926] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3900.919353] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3900.941133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3902.944525] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3902.945314] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3904.758632] Lustre: server umount lustre-MDT0000 complete [ 3907.870099] LustreError: 141844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787602793 with bad export cookie 9724132092632872820 [ 3907.871690] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3907.883534] LustreError: 141844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3908.206078] Lustre: server umount lustre-MDT0001 complete [ 3917.073923] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 3919.626638] Key type lgssc unregistered [ 3919.916587] LNet: 145801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3919.929951] LNetError: 145801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3919.954668] LNet: Removed LNI 192.168.206.116@tcp [ 3920.818743] Key type .llcrypt unregistered [ 3920.826249] Key type ._llcrypt unregistered [ 3941.238868] Key type ._llcrypt registered [ 3941.241400] Key type .llcrypt registered [ 3941.358895] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 3942.224988] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3942.297737] alg: No test for adler32 (adler32-zlib) [ 3943.309814] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 3943.513166] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 3945.151169] Key type lgssc registered [ 3946.380394] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3952.824143] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3952.857794] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3954.431463] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3958.863552] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3963.872803] LustreError: 147192:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3963.900682] LustreError: 147192:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3968.111968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3968.728757] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3973.012567] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3981.064310] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3987.090628] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3993.217086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3993.514332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3994.550115] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3999.462629] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 3999.679066] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4006.009726] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4011.801066] Lustre: *** cfs_fail_loc=159, val=0*** [ 4026.950478] Lustre: lustre-MDT0000: Client c612811d-0754-48ea-aebd-0ff41e25d133 (at 192.168.206.16@tcp) reconnecting [ 4033.008443] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4033.012792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4033.015155] LustreError: Skipped 1 previous similar message [ 4033.033154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4033.524878] Lustre: server umount lustre-OST0000 complete [ 4040.160373] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4040.166093] LustreError: Skipped 3 previous similar messages [ 4040.171929] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4040.192134] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4042.985202] Lustre: server umount lustre-MDT0000 complete [ 4045.793109] LustreError: 147923:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4045.831903] LustreError: 147923:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4046.800196] LustreError: 147173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787602932 with bad export cookie 12385639717002778564 [ 4046.818764] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4057.241461] Lustre: server umount lustre-MDT0001 complete [ 4070.081370] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4074.112951] Key type lgssc unregistered [ 4074.500479] LNet: 150857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4074.511853] LNetError: 150857:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4074.535812] LNet: Removed LNI 192.168.206.116@tcp [ 4075.873513] Key type .llcrypt unregistered [ 4075.876644] Key type ._llcrypt unregistered [ 4099.944347] Key type ._llcrypt registered [ 4099.946444] Key type .llcrypt registered [ 4100.040190] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4101.046887] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4101.111639] alg: No test for adler32 (adler32-zlib) [ 4102.142804] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 4102.368308] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4104.055183] Key type lgssc registered [ 4105.425416] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4112.606776] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4112.651840] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4114.162178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4118.437924] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4124.642893] LustreError: 152275:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4124.656297] LustreError: 152275:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4126.204740] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4126.747925] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4130.493872] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4137.104685] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4142.115382] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4148.334084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4148.731283] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4150.779041] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4154.251743] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4158.979423] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4161.882735] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4182.933210] Lustre: lustre-OST0000: evicting 2f6496e4-f872-422d-86ec-67c0c47fe1e5 (at 192.168.206.16@tcp) by administrative request [ 4212.192256] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4212.200204] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4212.240665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4215.269886] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4215.271325] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4215.277484] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4217.337539] Lustre: server umount lustre-OST0000 complete [ 4223.968125] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4223.983337] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4224.004644] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4224.011544] Lustre: Skipped 1 previous similar message [ 4227.492818] Lustre: server umount lustre-MDT0000 complete [ 4230.626698] LustreError: 153004:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4230.652116] LustreError: 153004:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4231.460302] LustreError: 152255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787603116 with bad export cookie 8999533696365730618 [ 4231.475932] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4231.794694] Lustre: server umount lustre-MDT0001 complete [ 4241.619989] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4244.715745] Key type lgssc unregistered [ 4245.039884] LNet: 156353:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4245.045373] LNetError: 156353:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4245.067522] LNet: Removed LNI 192.168.206.116@tcp [ 4245.906472] Key type .llcrypt unregistered [ 4245.909126] Key type ._llcrypt unregistered [ 4317.153193] Key type ._llcrypt registered [ 4317.154622] Key type .llcrypt registered [ 4317.273125] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4318.829471] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4319.000371] alg: No test for adler32 (adler32-zlib) [ 4320.201919] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 4320.546403] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4322.265794] Key type lgssc registered [ 4323.453896] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4331.551461] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4331.573619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4333.174891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4338.395566] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4342.756614] LustreError: 158841:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4342.780861] LustreError: 158841:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4346.458307] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4346.835614] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4351.418655] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4358.468476] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4363.692240] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4370.454222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4370.982757] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4372.042810] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4376.951510] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4378.229922] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4383.667901] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4392.930695] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4392.937825] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4392.955578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4393.955552] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4393.971825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4393.975472] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4394.005775] Lustre: Skipped 1 previous similar message [ 4398.459219] Lustre: server umount lustre-OST0000 complete [ 4403.168429] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4403.177159] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4403.206141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4408.595767] Lustre: server umount lustre-MDT0000 complete [ 4409.319360] LustreError: 158842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4409.341350] LustreError: 158842:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4413.017996] LustreError: 158822:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787603298 with bad export cookie 12697746624723849635 [ 4413.019152] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4413.043491] LustreError: 158822:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4413.463522] Lustre: server umount lustre-MDT0001 complete [ 4425.877680] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4429.209401] Key type lgssc unregistered [ 4429.577660] LNet: 162386:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4429.586616] LNetError: 162386:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4429.612134] LNet: Removed LNI 192.168.206.116@tcp [ 4430.488386] Key type .llcrypt unregistered [ 4430.489833] Key type ._llcrypt unregistered [ 4454.399964] Key type ._llcrypt registered [ 4454.401562] Key type .llcrypt registered [ 4454.493911] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4455.605602] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4455.734265] alg: No test for adler32 (adler32-zlib) [ 4456.886510] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 4457.208457] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4458.935984] Key type lgssc registered [ 4460.074213] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4475.530553] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 16:29:19 (1787603359) [ 4492.397637] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 4502.748904] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4540.041497] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4540.058585] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4541.373569] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4541.411242] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4541.501373] Lustre: lustre-MDT0000: new disk, initializing [ 4541.584550] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4541.600240] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4546.300853] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4557.168914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4557.296424] Lustre: 166654:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4557.369209] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4557.377947] Lustre: Skipped 1 previous similar message [ 4557.486245] Lustre: lustre-MDT0001: new disk, initializing [ 4557.574404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4557.618954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4557.641988] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4562.637726] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4572.146360] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4578.188742] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4585.718901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4586.145961] Lustre: lustre-OST0000: new disk, initializing [ 4586.153300] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4586.163350] Lustre: 168624:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4586.265748] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4592.193900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4592.203427] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4592.455192] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4593.727920] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4603.214557] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4618.208203] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4618.208332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4618.219378] LustreError: Skipped 1 previous similar message [ 4618.220076] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4618.262388] Lustre: Skipped 1 previous similar message [ 4619.439074] Lustre: server umount lustre-OST0000 complete [ 4623.328804] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4623.331255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4623.350869] Lustre: Skipped 1 previous similar message [ 4623.374472] Lustre: Skipped 2 previous similar messages [ 4628.457953] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4628.467266] Lustre: Skipped 1 previous similar message [ 4629.632391] Lustre: server umount lustre-MDT0000 complete [ 4633.475967] LustreError: 168623:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787603518 with bad export cookie 6598679006713180032 [ 4633.479300] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4633.496436] LustreError: 168623:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4633.876233] Lustre: server umount lustre-MDT0001 complete [ 4644.453159] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 4647.080274] Key type lgssc unregistered [ 4647.428992] LNet: 170531:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4647.445955] LNetError: 170531:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4647.487858] LNet: Removed LNI 192.168.206.116@tcp [ 4648.353815] Key type .llcrypt unregistered [ 4648.357786] Key type ._llcrypt unregistered [ 4672.107273] Key type ._llcrypt registered [ 4672.114577] Key type .llcrypt registered [ 4672.251638] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 4673.575205] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4673.602792] alg: No test for adler32 (adler32-zlib) [ 4674.723531] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 4675.031108] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 4676.727213] Key type lgssc registered [ 4678.551939] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4686.707589] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4686.735555] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4688.177315] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4692.295862] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4698.597861] LustreError: 171947:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4698.615518] LustreError: 171947:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4700.774626] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4701.130202] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4705.744104] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4712.831224] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4717.512525] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4723.279321] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4723.553808] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4728.718532] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 4732.795435] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4734.860794] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4741.863172] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4741.898634] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4741.932064] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4741.959652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.994556] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4742.865246] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4742.881333] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4742.888874] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4742.894372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.903220] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4743.803743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.548745] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4744.555976] Lustre: Skipped 1 previous similar message [ 4744.568672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4744.581727] Lustre: Skipped 1 previous similar message [ 4744.586058] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4744.590087] Lustre: Skipped 1 previous similar message [ 4744.592686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.601836] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4744.606500] Lustre: Skipped 1 previous similar message [ 4745.480272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.393184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.290931] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4747.296696] Lustre: Skipped 2 previous similar messages [ 4747.326919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4747.339156] Lustre: Skipped 2 previous similar messages [ 4747.349982] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4747.355331] Lustre: Skipped 2 previous similar messages [ 4747.360084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.366059] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4747.374112] Lustre: Skipped 2 previous similar messages [ 4748.306889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.062422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.722885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.598578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.346838] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4751.353939] Lustre: Skipped 4 previous similar messages [ 4751.363523] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4751.372168] Lustre: Skipped 4 previous similar messages [ 4751.375949] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4751.380603] Lustre: Skipped 4 previous similar messages [ 4751.384521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.395653] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4751.403235] Lustre: Skipped 4 previous similar messages [ 4752.098553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.787939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.505959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.274050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.199785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.974494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.841568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.794821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.873874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.658725] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4759.663438] Lustre: Skipped 9 previous similar messages [ 4759.677290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4759.687119] Lustre: Skipped 9 previous similar messages [ 4759.693304] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4759.698576] Lustre: Skipped 9 previous similar messages [ 4759.702263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.706582] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4759.709292] Lustre: Skipped 9 previous similar messages [ 4760.606249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.536363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.461151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.381192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.155834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.963920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.900119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.672413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.537508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.310505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.148968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.027674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.874544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.694828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.555238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.448548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.321624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.196516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.033220] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4776.053631] Lustre: Skipped 18 previous similar messages [ 4776.073089] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4776.085653] Lustre: Skipped 18 previous similar messages [ 4776.089395] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4776.096790] Lustre: Skipped 18 previous similar messages [ 4776.101648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.117702] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4776.123396] Lustre: Skipped 18 previous similar messages [ 4777.026768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.830054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.533140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.383984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.252215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.038368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.041725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.951038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.890170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.713764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.737169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.758763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.701346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.731562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.682127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.458204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.235865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.081749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.881710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.567462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.405531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.258727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.066657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.872649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.759366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.691499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.649637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.508868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.237258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.344784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.331591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.187278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.996139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.796766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.679643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.679697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.707942] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4808.722751] Lustre: Skipped 36 previous similar messages [ 4808.761210] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4808.791365] Lustre: Skipped 36 previous similar messages [ 4808.798585] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4808.805896] Lustre: Skipped 36 previous similar messages [ 4808.812853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.828836] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4808.839581] Lustre: Skipped 36 previous similar messages [ 4810.095889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.057308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.140290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.053592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.027857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.990588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.791115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.966981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.035686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.197056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.328373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.224543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.031162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.972156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.141314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.445808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.574710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.046666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.451896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.525283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.605342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.538168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.381423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.340023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.259612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.250222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.480858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.417081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.476186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.440787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.464370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.363102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.426447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.066528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.882706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.022688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.248595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.431610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.438545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.772590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.606585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.473267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.298095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.643071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.660535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.733984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.784836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.908511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.951256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.751300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.751124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.560046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.526909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.287407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.042201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.791334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.834510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.709621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.496907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.344728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.283255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.209666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.174608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.000617] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4873.020305] Lustre: Skipped 63 previous similar messages [ 4873.036776] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4873.048411] Lustre: Skipped 63 previous similar messages [ 4873.052967] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4873.061434] Lustre: Skipped 63 previous similar messages [ 4873.065766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.076619] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4873.082522] Lustre: Skipped 63 previous similar messages [ 4873.785065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.623264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.489294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.338409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.273176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.097504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.948724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.825973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.777318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.530271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.350737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.100448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.935588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.751918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.638548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.465357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.374362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.161471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.022653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.020916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.925135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.713900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.541113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.556152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.432146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.236890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.024807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.764400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.599590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.429736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.485453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.397719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.536714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.434396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.423269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.123942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.034970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.954763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.877348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.735802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.586594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.487922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.260545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.087966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.876417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.675341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.415537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.181149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.196112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.069639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.876129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.857725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.730466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.380187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.202440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.887886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.660216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.445026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.238709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.140084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.016224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.872506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.587658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.464970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.399537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.262576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.085414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.928798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.825185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.704362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.413155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.563656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.409305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.292899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.107368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.880405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.623492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.415281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.143075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.987114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.652224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.567269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.406843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.204824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.006970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.868495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.562315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.453921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.254775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.515259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.682057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.680479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.657431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.573650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.375538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.113606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.083151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.988271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.899919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.942981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.819145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.674516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.464099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.216088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.087067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.011753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.875540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.811702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.874678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.780877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.873336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.779575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.695607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.579793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.556542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.250770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.093673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.977678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.824181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.705793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.584660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.310586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.029955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.832429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.778317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.716296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.534672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.543115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.667823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.781110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.569566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.431825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.176994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.893082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.640507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.436402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.227776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.022705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.959585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.957172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.801144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.607358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.506366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.238439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.230271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.148806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.967264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.953504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.897507] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5001.901372] Lustre: Skipped 148 previous similar messages [ 5001.913478] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5001.925829] Lustre: Skipped 148 previous similar messages [ 5001.935845] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5001.942940] Lustre: Skipped 148 previous similar messages [ 5001.947619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.958740] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5001.964139] Lustre: Skipped 148 previous similar messages [ 5002.890939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.665519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.641384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.523464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.541489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.479305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.425134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.329967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.472859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.354449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.323145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.272160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.215375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.150707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.115914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.947425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.786863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.002534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.140314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.043345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.894357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.693727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.597893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.552911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.711233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.797572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.894165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.956226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.873557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.927321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.796639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.547633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.463477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.389969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.271171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.062457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.099405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.160136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.320856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.164378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.193332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.018313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.093618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.949419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.887316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.720476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.698368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.655769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.655454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.662425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.648074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.647558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.556387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.762079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.830550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.844843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.695364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.661587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.607264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.414686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.569053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.760877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.226411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.361217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.756845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.969078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.606257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.407414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.616330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.770103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.726182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.355543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.194286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.116892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.193108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.499074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.559903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.795567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.904698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.188656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.109752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.498468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.643847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.871194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.413771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.500302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.485287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.790546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.426777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.021450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.892644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.221977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.441698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.549520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.657229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.057764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.054188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.144895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.098678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.271895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.384313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.309895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.364556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.262773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.371617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.298693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.403449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.242325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.298779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.658656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.781294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.701129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.545237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.312454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.261261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.092446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.925660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.862318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.920946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.892020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.772536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.629528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.481711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.723639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.809216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.716436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.375540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.263462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.124568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.041257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.814427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.667393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.664439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.558602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.460556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.314187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.149709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.974243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.861867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.795332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.682448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.594493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.556696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.415742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.408383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.497879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.445416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.438487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.401015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.322855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.229447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.278707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.137220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.020690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.967247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.871684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.831871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.722391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.589265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.499664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.366993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.307628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.224498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.059911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.877163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.687651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.622322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.616118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.514868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.247088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.881316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.568907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.193738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.888384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.720602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.332972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.058895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.760237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.489237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.191242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.900331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.593484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.327411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.105857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.899746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.738319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.959361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.836898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.647829] LustreError: 171967:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5187.657459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.658393] LustreError: 171967:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5188.363208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.114484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.834613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.519793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.202419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.952351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.609218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.412890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.391767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.229255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.118558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.900477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.812157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.658638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.547982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.398697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.289917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.112236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.934585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.662166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.608883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.636860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.704335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.645142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.844953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.660874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.508830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.354805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.095227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.084544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.955642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.899941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.745688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.636980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.411422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.248365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.052694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.964184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.853722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.662399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.440895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.329731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.138780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.993700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.699670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.620258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.573401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.539703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.446293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.375942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.095233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.966468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.722048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.717754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.383326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.149309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.084691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.096369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.079700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.002188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.049395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.052370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.940298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.815670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.816950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.843321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.834510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.937971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.087318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.986353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.756878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.835483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.056569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.971285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.151357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.407830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.485881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.303390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.316558] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5258.318994] Lustre: Skipped 267 previous similar messages [ 5258.329734] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5258.337295] Lustre: Skipped 267 previous similar messages [ 5258.339247] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5258.341605] Lustre: Skipped 267 previous similar messages [ 5258.344373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.350968] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5258.356217] Lustre: Skipped 267 previous similar messages [ 5259.148897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.092423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.382252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.106647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.940709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.825595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.761607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.757057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.740727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.704853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.605694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.517605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.484150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.332024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.208888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.003035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.946300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.934407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.898712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.852727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.862886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.029385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.874037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.807395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.542780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.256823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.085147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.128077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.095353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.286581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.241785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.980859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.809202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.597559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.363889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.252685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.963527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.741208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.458918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.229632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.096609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.990221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.800549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.541024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.383199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.303354] LustreError: 171967:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5299.312396] LustreError: 171967:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5299.322048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.213320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.971761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.775970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.510373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.385680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.336976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.310773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.384860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.579417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.594866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.825410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.911075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.781709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.705880] LustreError: 171967:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5312.721074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.721603] LustreError: 171967:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5313.572336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.620403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.716109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.697463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.624131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.363446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.255972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.193813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.265366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.126464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.967638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.992333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.836572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.881749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.623819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.511963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.264935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.998592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.969175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.831838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.797299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.685796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.627785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.504732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.404636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.531663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.396953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.280384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.344411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.212064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.239913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.233371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.972570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.785036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.645578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.595218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.480337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.442938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.384313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.345796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.371157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.617405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.502562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.424121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.382473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.347626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.232338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.366941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.269037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.252144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.307815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.533368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.386533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.492316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.467206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.273417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.106039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.917246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.773754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.546201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.503181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.483205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.802430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.943340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.042143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.248533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.177698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.136138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.233782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.036221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.909223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.909712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.787030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.684851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.714583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.764582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.747429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.825601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.826083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.895901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.862385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.020227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.907531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.822875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.811165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.903144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.787207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.909074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.770543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.829104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.781974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.984428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.041948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.072582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.068164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.830595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.831519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.758814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.092483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.241117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.250705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.162851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.923618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.840265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.753473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.670312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.538929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.509271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.446526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.440798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.210108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.167581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.043777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.005511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.992732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.977161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.689310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.536363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.268035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.103888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.091168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.963544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.723597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.510038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.421850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.457813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.251491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.165773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.915219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.720303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.494946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.348604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.192726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.260171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.291590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.455062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.263087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.267742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.053369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.928722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.792815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.750844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.556663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.386298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.174289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.104413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.044566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.006720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.969324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.920753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.695622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.634631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.571352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.558595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.521157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.407946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.282483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.446052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.364502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.289690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.310476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.249470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.116517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.966033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.836541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.665220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.612960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.486787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.439617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.390987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.205979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.989675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.780069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.728742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.478176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.330285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.181453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.078793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.865086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.624618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.418124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.226688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.992060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.860506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.674500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.468080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.441598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.337860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.288399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.015249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.811464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.848957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.671317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.512303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.381381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.001490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.891490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.687987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.527749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.372965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.201428] LustreError: 171967:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5498.212637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.213545] LustreError: 171967:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5499.159492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.031061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.829824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.785076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.740392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.714058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.585209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.520198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.332295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.244106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.168661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.176738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.185693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.203155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.236825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.315179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.437864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.272084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.133268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.944676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.766582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.578392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.370947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.168414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.065918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.004239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.884148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.790396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.631975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.388711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.194103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.908801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.745300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.597376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.480655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.313877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.210699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.933873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.765234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.481451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.227568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.925116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.649436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.409497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.189571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.906078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.719381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.740308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.650553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.627696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.584097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.398346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.414554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.349908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.356604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.215052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.084550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.939935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.858967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.704877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.526781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.537386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.311947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.166077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.234802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.216416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.192454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.081220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.003185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.777423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.799721] LustreError: 171967:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5560.811223] LustreError: 171967:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5560.812947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.624189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.527514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.314194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.458775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.489758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.323181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.296047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.221064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.164976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.910461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.775772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.682071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.463879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.313371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.303990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.167495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.485075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.705786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.741179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.634637] LustreError: 171967:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5579.643540] LustreError: 171967:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5579.651396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.479992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.509851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.577190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.588490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.547108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.569388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.392374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.688984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.809058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.687335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.613051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.496471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.415462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.432490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.318823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.168486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.055455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.976622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.830418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.797025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.602627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.417896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.334810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.296488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.012341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.750035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.504600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.368336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.188291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.039465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.846993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.861204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.529298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.329231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.033338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.931254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.877855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.709796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.710179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.846106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.747561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.609521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.612444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.542898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.528947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.607245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.533545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.453966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.422307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.358733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.389815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.319477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.297141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.262557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.947655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.865090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.856789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.695758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.796604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.788548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.761082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.940792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.086559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.223746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.338323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.082252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.838129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.685332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.589850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.647288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.557477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.499194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.453842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.482975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.620671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.517161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.556742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.506577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.254441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.120739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.939747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.820912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.796178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.608069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.499536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.306677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.175862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.187472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.211550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.995736] LustreError: 173767:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5971.008311] LustreError: 171967:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6409.189624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6409.201979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6409.205249] Lustre: Skipped 442 previous similar messages [ 6409.219932] Lustre: Skipped 1 previous similar message [ 6413.693235] Lustre: server umount lustre-OST0000 complete [ 6419.431929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6419.444397] Lustre: Skipped 1 previous similar message [ 6423.695837] Lustre: server umount lustre-MDT0000 complete [ 6424.545739] LustreError: 171947:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6424.563899] LustreError: 171947:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6426.955515] LustreError: 171927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787605312 with bad export cookie 1705661779715736415 [ 6426.967668] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6427.344979] Lustre: server umount lustre-MDT0001 complete [ 6436.600085] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 6439.194491] Key type lgssc unregistered [ 6439.440772] LNet: 227168:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6439.447244] LNetError: 227168:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6439.466335] LNet: Removed LNI 192.168.206.116@tcp [ 6440.152357] Key type .llcrypt unregistered [ 6440.155465] Key type ._llcrypt unregistered [ 6461.794235] Key type ._llcrypt registered [ 6461.796487] Key type .llcrypt registered [ 6461.931666] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 6462.872495] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6463.131775] alg: No test for adler32 (adler32-zlib) [ 6464.194777] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 6464.491612] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 6466.256048] Key type lgssc registered [ 6467.622453] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6475.689796] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6475.707745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6477.044938] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6480.911798] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6487.523386] LustreError: 228583:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6487.540537] LustreError: 228583:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6488.198970] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6488.582839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6491.820049] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6498.176677] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6502.851616] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6509.198916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6509.608293] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6515.650245] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6521.996292] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6522.361371] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6528.197931] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6530.065780] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6530.080407] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6530.090332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.096559] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6530.099527] LustreError: 228603:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6532.588689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6532.608710] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6532.612887] Lustre: Skipped 1 previous similar message [ 6532.623502] Lustre: Skipped 1 previous similar message [ 6535.227272] Lustre: lustre-OST0000: Not available for connect from 192.168.206.16@tcp (stopping) [ 6537.699655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6537.706536] Lustre: Skipped 1 previous similar message [ 6540.309471] Lustre: lustre-OST0000: Not available for connect from 192.168.206.16@tcp (stopping) [ 6545.428595] Lustre: lustre-OST0000: Not available for connect from 192.168.206.16@tcp (stopping) [ 6545.431219] Lustre: Skipped 2 previous similar messages [ 6546.399114] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6546.574273] Lustre: server umount lustre-OST0000 complete [ 6550.159115] LustreError: 228603:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6579.394562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6579.671674] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6585.618689] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6591.490459] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6598.135934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.173929] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.178238] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6598.204323] Lustre: Skipped 1 previous similar message [ 6603.123246] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 9 sec [ 6608.937109] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9d85c4d03800.ost_server_uuid 50 [ 6610.618646] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9d85c4d03800.ost_server_uuid in FULL state after 0 sec [ 6616.032647] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6616.038576] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6616.046185] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6617.518980] Lustre: server umount lustre-OST0000 complete [ 6621.665074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6621.672932] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6626.561167] Lustre: server umount lustre-MDT0000 complete [ 6628.833646] LustreError: 228583:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6628.857033] LustreError: 228583:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6629.769127] LustreError: 230426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787605515 with bad export cookie 4156276159372326623 [ 6629.777973] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6629.780161] LustreError: 230426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6640.240636] Lustre: server umount lustre-MDT0001 complete [ 6651.253361] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 6655.721596] Key type lgssc unregistered [ 6656.221785] LNet: 233816:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6656.233825] LNetError: 233816:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6656.276262] LNet: Removed LNI 192.168.206.116@tcp [ 6657.682040] Key type .llcrypt unregistered [ 6657.689425] Key type ._llcrypt unregistered [ 6694.539750] Key type ._llcrypt registered [ 6694.541925] Key type .llcrypt registered [ 6694.652791] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 6695.868479] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6695.892971] alg: No test for adler32 (adler32-zlib) [ 6696.874353] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 6697.049053] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 6698.683224] Key type lgssc registered [ 6700.104491] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6710.654531] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6710.713092] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6712.536812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6718.694057] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6722.024129] LustreError: 235549:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6722.049799] LustreError: 235549:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6727.144956] LustreError: 235550:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6728.878309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6729.290383] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6734.552631] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6737.576971] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6743.481918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6743.843894] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6749.143171] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6756.754789] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6756.931417] Lustre: lustre-OST0001: new disk, initializing [ 6756.934474] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6756.939571] Lustre: 237857:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6757.025037] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6758.088078] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6758.716522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6758.737377] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6758.848935] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6762.654674] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6773.095888] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6778.719921] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6790.364349] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 6790.383370] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6790.397323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6792.683889] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6792.694751] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6792.703488] Lustre: Skipped 2 previous similar messages [ 6792.718901] Lustre: Skipped 3 previous similar messages [ 6796.653068] Lustre: server umount lustre-MDT0000 complete [ 6797.793632] LustreError: 235550:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6797.815560] LustreError: 235550:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6800.043283] LustreError: 236684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787605685 with bad export cookie 9178804743513603154 [ 6800.048718] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6800.055136] LustreError: 236684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6800.063461] LustreError: 236288:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 6800.458217] Lustre: server umount lustre-MDT0001 complete [ 6814.140470] Lustre: server umount lustre-OST0000 complete [ 6828.232482] Lustre: server umount lustre-OST0001 complete [ 6851.876975] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 6862.618629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6863.115784] LustreError: 240969:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6863.184119] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6868.450192] LustreError: 240970:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6868.698574] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6878.274076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6878.876022] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6883.856294] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6886.658355] Lustre: 242110:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6893.102090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6898.845853] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6903.724561] LustreError: 242465:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6903.740675] LustreError: 242465:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6903.762390] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 6906.854812] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6907.159722] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6907.165570] Lustre: Skipped 1 previous similar message [ 6908.230410] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 6913.081285] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6920.743676] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6924.159221] Lustre: 243980:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6933.993731] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6934.002944] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6934.011508] Lustre: Skipped 2 previous similar messages [ 6934.014831] Lustre: Skipped 1 previous similar message [ 6934.171390] Lustre: server umount mylustre-MDT0000 complete [ 6937.758784] LustreError: 240946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787605823 with bad export cookie 9178804743513606010 [ 6937.764230] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6937.772962] LustreError: 240946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6939.105576] LustreError: 240966:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6939.109165] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6939.132223] LustreError: 240966:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6939.151491] Lustre: Skipped 1 previous similar message [ 6939.161626] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6939.180827] Lustre: Skipped 3 previous similar messages [ 6944.164260] Lustre: server umount mylustre-MDT0001 complete [ 6948.137407] Lustre: server umount mylustre-OST0000 complete [ 6951.639691] Lustre: server umount mylustre-OST0001 complete [ 6970.376953] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 6978.886915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6979.358863] LustreError: 246286:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6979.375746] LustreError: 246286:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6979.430620] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6983.627124] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6991.473724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6996.000199] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 6999.038136] Lustre: 247428:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7005.511580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7014.230355] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7017.104226] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 7024.708906] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7026.134134] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7031.845667] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7034.628549] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7043.912524] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7047.182658] Lustre: 249300:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7057.400532] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7057.402872] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7057.432968] Lustre: Skipped 4 previous similar messages [ 7057.450116] Lustre: Skipped 5 previous similar messages [ 7060.519217] Lustre: server umount tfs-MDT0000 complete [ 7062.503517] LustreError: 246283:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7062.530996] LustreError: 246283:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 7064.159725] LustreError: 247030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787605949 with bad export cookie 9178804743513607963 [ 7064.167837] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7064.170632] LustreError: 247030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7064.490677] Lustre: server umount tfs-MDT0001 complete [ 7077.958876] Lustre: server umount tfs-OST0000 complete [ 7092.554817] Lustre: server umount tfs-OST0001 complete [ 7115.571428] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7126.366586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7126.890073] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7126.901229] Lustre: Skipped 3 previous similar messages [ 7131.382369] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7140.721938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7146.392640] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7149.424275] Lustre: 252695:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7157.990661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7164.658310] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7165.688640] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 7172.097811] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7173.239054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7178.761390] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7179.696413] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7187.105589] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7190.604861] Lustre: 254566:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7196.934967] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 17:14:40 (1787606080) [ 7212.513463] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7212.528221] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7212.555563] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7218.996406] Lustre: server umount lustre-MDT0000 complete [ 7219.687828] LustreError: 251548:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7219.709480] LustreError: 251548:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 7222.704822] LustreError: 251530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787606108 with bad export cookie 9178804743513609881 [ 7222.707928] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7222.731167] LustreError: 251530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7232.935179] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7232.944647] Lustre: Skipped 9 previous similar messages [ 7237.087738] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7237.339793] Lustre: server umount lustre-MDT0001 complete [ 7247.805736] Lustre: server umount lustre-OST0000 complete [ 7257.735414] Lustre: server umount lustre-OST0001 complete [ 7264.585537] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 7274.336951] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7311.817727] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7312.166640] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7312.212161] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7312.400440] Lustre: lustre-MDT0000: new disk, initializing [ 7312.563207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7312.568498] Lustre: Skipped 3 previous similar messages [ 7312.587538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7317.945053] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7330.534956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7330.719836] Lustre: 257749:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7330.772638] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7330.776927] Lustre: Skipped 1 previous similar message [ 7330.861602] Lustre: lustre-MDT0001: new disk, initializing [ 7331.071803] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7331.092609] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7336.276448] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7348.112515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7348.443730] Lustre: lustre-OST0000: new disk, initializing [ 7348.451720] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7348.460380] Lustre: 259331:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7349.792764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7349.816239] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7350.005126] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7356.382923] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7369.437413] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7369.785111] Lustre: lustre-OST0001: new disk, initializing [ 7369.793468] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7369.805068] Lustre: 260202:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7371.719732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7371.738922] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7371.918830] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7378.082987] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7392.989907] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 17:17:56 (1787606276) [ 7409.988955] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 17:18:13 (1787606293) [ 7427.040772] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7427.046848] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7427.048067] LustreError: Skipped 1 previous similar message [ 7427.049795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7427.074741] Lustre: Skipped 6 previous similar messages [ 7431.569912] Lustre: server umount lustre-OST0000 complete [ 7441.707987] Lustre: server umount lustre-MDT0000 complete [ 7446.045866] LustreError: 257740:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787606331 with bad export cookie 9178804743513611659 [ 7446.053778] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7446.064527] LustreError: 257740:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7452.428722] Lustre: server umount lustre-MDT0001 complete [ 7464.505104] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 7539.003979] Lustre: server umount lustre-OST0001 complete [ 7546.727880] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 7564.417065] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7602.676536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7602.846133] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7602.882608] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7602.948847] Lustre: lustre-MDT0000: new disk, initializing [ 7603.034174] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7603.044147] Lustre: Skipped 3 previous similar messages [ 7603.057967] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7607.103636] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7617.466273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7617.629393] Lustre: 265062:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7617.944440] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7622.701470] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7631.475479] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7636.547275] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7642.695715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7642.902588] Lustre: lustre-OST0000: new disk, initializing [ 7642.905226] Lustre: Skipped 1 previous similar message [ 7642.909285] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7642.912993] Lustre: Skipped 2 previous similar messages [ 7642.921687] Lustre: 267031:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7644.380264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7644.387391] Lustre: Skipped 1 previous similar message [ 7644.399277] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7644.502672] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7648.746407] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7657.547990] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7665.123386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7665.133299] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7665.147321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7665.155521] Lustre: Skipped 5 previous similar messages [ 7665.161929] Lustre: Skipped 8 previous similar messages [ 7669.825223] Lustre: server umount lustre-OST0000 complete [ 7674.337057] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7674.344645] LustreError: Skipped 1 previous similar message [ 7680.264605] Lustre: server umount lustre-MDT0000 complete [ 7680.481292] LustreError: 265074:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7680.503099] LustreError: 265074:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 16 previous similar messages [ 7683.812631] LustreError: 265057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787606569 with bad export cookie 9178804743513613129 [ 7683.815410] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7683.834128] LustreError: 265057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7684.488410] Lustre: server umount lustre-MDT0001 complete [ 7697.822600] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 7701.068625] Key type lgssc unregistered [ 7701.484937] LNet: 268796:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7701.495450] LNetError: 268796:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7701.512712] LNet: Removed LNI 192.168.206.116@tcp [ 7702.463348] Key type .llcrypt unregistered [ 7702.467239] Key type ._llcrypt unregistered [ 7729.139389] Key type ._llcrypt registered [ 7729.141611] Key type .llcrypt registered [ 7729.240514] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7730.143058] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7730.215799] alg: No test for adler32 (adler32-zlib) [ 7731.344758] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 7731.578659] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 7733.263174] Key type lgssc registered [ 7734.260399] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7741.669644] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7741.698718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7743.125880] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7747.279336] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7752.685586] LustreError: 270260:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7752.708106] LustreError: 270260:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7755.482476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7755.834125] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7759.870950] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7766.649761] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7771.960414] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7778.249170] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7778.654902] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7784.489438] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7787.896329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7791.578747] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7792.503202] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7796.209425] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7796.218589] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7796.240273] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7798.241284] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7798.247462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7798.272328] Lustre: Skipped 1 previous similar message [ 7800.516946] Lustre: server umount lustre-OST0000 complete [ 7806.946540] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7806.958435] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7806.995852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7811.942216] Lustre: server umount lustre-MDT0000 complete [ 7813.602685] LustreError: 270988:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7813.628728] LustreError: 270988:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7816.292194] LustreError: 270242:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787606701 with bad export cookie 6913599389066031082 [ 7816.302290] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7816.305760] LustreError: 270242:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7816.658739] Lustre: server umount lustre-MDT0001 complete [ 7828.344963] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 7830.815674] Key type lgssc unregistered [ 7831.084659] LNet: 273847:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7831.097313] LNetError: 273847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7831.112816] LNet: Removed LNI 192.168.206.116@tcp [ 7831.790789] Key type .llcrypt unregistered [ 7831.793265] Key type ._llcrypt unregistered [ 7847.206923] Key type ._llcrypt registered [ 7847.210327] Key type .llcrypt registered [ 7847.339328] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7848.327850] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7848.425443] alg: No test for adler32 (adler32-zlib) [ 7849.442093] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 7849.676070] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 7851.375242] Key type lgssc registered [ 7852.302897] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7860.556639] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7860.571757] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7876.127495] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7881.248508] LustreError: 275043:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.116@tcp: failed processing log, type 4: rc = -110 [ 7907.871392] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7913.600977] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 7920.071811] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7922.105606] Lustre: server umount lustre-OST0000 complete [ 7935.241856] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 7937.383074] Key type lgssc unregistered [ 7937.616680] LNet: 276680:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7937.620723] LNetError: 276680:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7937.636667] LNet: Removed LNI 192.168.206.116@tcp [ 7938.407776] Key type .llcrypt unregistered [ 7938.412756] Key type ._llcrypt unregistered [ 7976.594736] Key type ._llcrypt registered [ 7976.596080] Key type .llcrypt registered [ 7976.695282] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 7977.915947] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7977.953270] alg: No test for adler32 (adler32-zlib) [ 7979.029253] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 7979.325944] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 7981.007198] Key type lgssc registered [ 7982.059608] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7994.408903] loop0: detected capacity change from 0 to 204800000 [ 7994.597540] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7994.616018] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7994.927689] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7996.149046] loop0: detected capacity change from 0 to 204800000 [ 7996.319462] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7996.330052] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7996.694509] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7997.759930] loop0: detected capacity change from 0 to 204800000 [ 7997.970642] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7997.990714] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7998.119849] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7999.129934] loop0: detected capacity change from 0 to 204800000 [ 7999.229439] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8000.286587] loop0: detected capacity change from 0 to 204800000 [ 8000.340098] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8004.016484] loop0: detected capacity change from 0 to 204800000 [ 8004.071326] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8007.681803] loop0: detected capacity change from 0 to 204800000 [ 8007.708347] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8011.543231] loop0: detected capacity change from 0 to 204800000 [ 8011.581871] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8015.113211] loop0: detected capacity change from 0 to 204800000 [ 8015.189097] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8015.243380] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8015.309460] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8015.323021] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8015.340772] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8015.351113] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8015.360960] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8015.375403] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8015.393785] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8015.405707] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8015.414715] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8015.424788] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8015.437144] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8015.453749] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8015.462528] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8015.481888] Lustre: 280097:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8020.675376] Lustre: server umount MGS complete [ 8021.516534] loop0: detected capacity change from 0 to 204800000 [ 8021.635759] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8022.234214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8022.273646] Lustre: lustre-MDT0000: Aborting client recovery [ 8023.325936] loop1: detected capacity change from 0 to 204800000 [ 8023.570932] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8023.712342] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8023.727832] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8023.749471] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8023.768490] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8023.782714] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8023.804348] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8023.820298] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8023.851143] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8024.050751] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8024.101702] Lustre: lustre-MDT0001: Aborting client recovery [ 8025.236623] loop2: detected capacity change from 0 to 204800000 [ 8025.437849] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8025.526929] Lustre: 280610:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8025.542883] Lustre: 280610:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8025.559048] Lustre: 280610:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8025.585707] Lustre: 280610:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8025.594908] Lustre: 280610:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8025.610669] Lustre: 280610:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8025.623069] Lustre: 280610:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8025.822098] Lustre: Found index 0 for lustre-OST0000, updating log [ 8025.889720] Lustre: 280629:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8025.979740] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8026.036338] Lustre: lustre-OST0000: Aborting recovery [ 8027.017539] loop3: detected capacity change from 0 to 204800000 [ 8027.248226] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8027.330366] Lustre: 280695:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8027.340720] Lustre: 280695:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8027.347222] Lustre: 280695:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8027.353668] Lustre: 280695:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8027.362374] Lustre: 280695:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8027.369139] Lustre: 280695:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8027.378774] Lustre: 280695:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8027.392772] Lustre: Found index 1 for lustre-OST0001, updating log [ 8027.415963] Lustre: 280700:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8027.475347] Lustre: lustre-OST0001: Aborting recovery [ 8032.616850] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8032.619489] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8032.685657] LustreError: 280474:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 11, retries 0, failed: rc = -108 [ 8032.690591] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8032.715680] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8034.664491] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8034.673310] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8036.628985] LustreError: 280440:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3884us: rc = -2 [ 8037.217555] LustreError: 280440:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 4089us: rc = -2 [ 8037.249475] LustreError: 280440:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8050.145284] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8050.146644] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8050.164632] Lustre: Skipped 2 previous similar messages [ 8050.180070] Lustre: Skipped 3 previous similar messages [ 8055.264560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8055.277117] Lustre: Skipped 3 previous similar messages [ 8056.181368] Lustre: server umount lustre-MDT0000 complete [ 8057.012084] LustreError: 280423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787606942 with bad export cookie 8241394481728351341 [ 8057.012954] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8057.030022] LustreError: 280423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8060.384482] LustreError: 280989:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8060.389449] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8060.395034] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8060.412443] LustreError: 280989:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8060.443022] Lustre: Skipped 2 previous similar messages [ 8062.437532] LustreError: 280979:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8062.440073] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8062.459218] LustreError: 280979:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8062.467140] Lustre: Skipped 2 previous similar messages [ 8063.561762] Lustre: server umount lustre-MDT0001 complete [ 8070.861823] Lustre: server umount lustre-OST0000 complete [ 8077.992296] Lustre: server umount lustre-OST0001 complete [ 8086.726466] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:29:30 (1787606970) [ 8115.509216] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 8124.874154] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8160.149588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8160.451380] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8160.518249] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8160.643912] Lustre: lustre-MDT0000: new disk, initializing [ 8160.731052] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8160.742467] Lustre: Skipped 1 previous similar message [ 8160.771991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8165.104463] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8175.320883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8175.457366] Lustre: 284769:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8175.566826] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8175.577251] Lustre: Skipped 1 previous similar message [ 8175.692424] Lustre: lustre-MDT0001: new disk, initializing [ 8175.813436] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8175.850356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8175.868121] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8179.620772] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8188.194135] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8193.119935] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8199.236265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8199.517981] Lustre: lustre-OST0000: new disk, initializing [ 8199.524407] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8199.532889] Lustre: 286738:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8199.616811] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8205.310710] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8205.906626] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8205.930894] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8206.040403] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8214.069731] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8218.819619] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8232.673470] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8241.586256] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8298.474781] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8298.478248] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8298.482161] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8298.495572] Lustre: Skipped 1 previous similar message [ 8300.719561] Lustre: server umount lustre-OST0000 complete [ 8308.705749] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8308.707437] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8308.722513] Lustre: Skipped 1 previous similar message [ 8308.733661] Lustre: Skipped 2 previous similar messages [ 8310.684971] Lustre: server umount lustre-MDT0000 complete [ 8313.824806] LustreError: 285615:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8313.839937] LustreError: 285615:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8314.030031] LustreError: 284760:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787607199 with bad export cookie 8241394481728354043 [ 8314.039665] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8314.043153] LustreError: 284760:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8314.385542] Lustre: server umount lustre-MDT0001 complete [ 8322.619439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8327.214751] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8332.380443] Lustre: server umount MGS complete [ 8344.133492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8344.683554] LustreError: 290815:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8344.700776] LustreError: 290828:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8344.737525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8350.179799] LustreError: 290816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8350.183645] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8359.864820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8365.743920] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8373.282720] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8378.467896] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8384.536515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8384.963454] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8384.970910] Lustre: Skipped 1 previous similar message [ 8390.954451] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8398.151806] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8432.101429] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8432.112114] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8432.118557] LustreError: Skipped 2 previous similar messages [ 8432.119626] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8432.151421] Lustre: Skipped 1 previous similar message [ 8433.921020] Lustre: server umount lustre-OST0000 complete [ 8442.343540] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8442.345254] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8444.109444] Lustre: server umount lustre-MDT0000 complete [ 8447.457179] LustreError: 290816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8447.481319] LustreError: 290816:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8447.589859] LustreError: 293652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787607332 with bad export cookie 8241394481728355884 [ 8447.595472] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8447.600740] LustreError: 293652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8447.925954] Lustre: server umount lustre-MDT0001 complete [ 8458.615319] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 8461.551869] Key type lgssc unregistered [ 8461.875771] LNet: 294761:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8461.886459] LNetError: 294761:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8461.907252] LNet: Removed LNI 192.168.206.116@tcp [ 8462.754421] Key type .llcrypt unregistered [ 8462.756436] Key type ._llcrypt unregistered [ 8501.860137] Key type ._llcrypt registered [ 8501.861631] Key type .llcrypt registered [ 8501.967760] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 8514.597423] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8515.373405] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8515.523931] alg: No test for adler32 (adler32-zlib) [ 8516.635825] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 8516.934165] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 8518.631911] Key type lgssc registered [ 8519.661818] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8552.408675] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8552.431353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8553.612057] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8553.644607] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8553.742292] Lustre: lustre-MDT0000: new disk, initializing [ 8553.826280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8553.839510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8557.730757] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8568.004089] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8568.147932] Lustre: 298701:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8568.190608] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8568.193777] Lustre: Skipped 1 previous similar message [ 8568.242862] Lustre: lustre-MDT0001: new disk, initializing [ 8568.329665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8568.380792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8568.389698] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8572.745390] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8581.637623] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8586.709479] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8592.619500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8592.971708] Lustre: lustre-OST0000: new disk, initializing [ 8592.977850] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8592.984198] Lustre: 300670:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8593.082859] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8599.242491] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8601.190428] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8601.204079] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8601.369416] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8608.181686] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8615.600317] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8627.752991] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8632.457646] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8638.146682] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8683.497062] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8683.506117] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8683.509795] LustreError: Skipped 1 previous similar message [ 8683.510767] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8683.544288] Lustre: Skipped 1 previous similar message [ 8687.393184] Lustre: server umount lustre-OST0000 complete [ 8693.737712] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8693.750836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8693.762147] Lustre: Skipped 1 previous similar message [ 8693.773156] Lustre: Skipped 2 previous similar messages [ 8697.722690] Lustre: server umount lustre-MDT0000 complete [ 8701.142950] LustreError: 298692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787607586 with bad export cookie 1785130110780127297 [ 8701.146196] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8701.150622] LustreError: 298692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8701.392205] Lustre: server umount lustre-MDT0001 complete [ 8709.373087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8714.057358] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8719.421041] Lustre: server umount MGS complete [ 8731.802531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8732.423591] LustreError: 304490:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8732.444732] LustreError: 304490:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8732.462774] LustreError: 304503:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8732.549523] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8736.764811] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8737.768243] LustreError: 304491:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8741.860054] LustreError: 304490:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8744.877404] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8745.215065] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8749.347609] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8757.256751] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8762.502540] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8768.860408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8769.097577] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8775.092372] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8783.021774] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8808.432922] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8808.447426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8808.460812] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8808.478137] Lustre: Skipped 1 previous similar message [ 8813.539252] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8814.115550] Lustre: server umount lustre-OST0000 complete [ 8818.661494] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8818.671549] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8818.685801] Lustre: Skipped 2 previous similar messages [ 8818.707632] Lustre: Skipped 2 previous similar messages [ 8826.389562] Lustre: server umount lustre-MDT0000 complete [ 8828.901081] LustreError: 305739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8828.912438] LustreError: 305739:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8830.139632] LustreError: 304471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787607715 with bad export cookie 1785130110780129236 [ 8830.140956] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8830.162933] LustreError: 304471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8830.658505] Lustre: server umount lustre-MDT0001 complete [ 8842.086808] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [ 8845.084512] Key type lgssc unregistered [ 8845.370854] LNet: 308437:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8845.387729] LNetError: 308437:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8845.400743] LNet: Removed LNI 192.168.206.116@tcp [ 8846.175858] Key type .llcrypt unregistered [ 8846.179732] Key type ._llcrypt unregistered [ 8865.664239] Key type ._llcrypt registered [ 8865.666701] Key type .llcrypt registered [ 8865.793559] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 8878.993942] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8879.993489] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8880.068374] alg: No test for adler32 (adler32-zlib) [ 8881.071832] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [ 8881.323648] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [ 8883.007360] Key type lgssc registered [ 8883.997548] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8906.429023] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 8920.004152] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8920.018063] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8921.222304] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8921.239992] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8921.314135] Lustre: lustre-MDT0000: new disk, initializing [ 8921.363762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8921.386970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8925.808339] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 8934.676181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8934.881898] Lustre: lustre-OST0000: new disk, initializing [ 8934.887716] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8934.895375] Lustre: Skipped 1 previous similar message [ 8934.900365] Lustre: 312466:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8934.971653] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8938.089093] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8938.096222] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8938.130571] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8941.201519] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9711.074468] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9711.097156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9716.197076] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9717.043494] Lustre: server umount lustre-MDT0000 complete [ 9720.986354] LustreError: 311654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787608606 with bad export cookie 17043517889013050578 [ 9720.994968] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9721.001152] LustreError: 311654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9721.197573] Lustre: server umount lustre-OST0000 complete [ 9749.376954] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [ 9761.886602] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [ 9795.411041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9795.685764] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9795.747073] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9795.864219] Lustre: lustre-MDT0000: new disk, initializing [ 9795.950060] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9795.972445] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9800.112476] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9809.135908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9809.251269] Lustre: 316791:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9809.297100] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9809.300535] Lustre: Skipped 1 previous similar message [ 9809.355533] Lustre: lustre-MDT0001: new disk, initializing [ 9809.420497] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9809.462792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9809.477081] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9813.156112] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9821.888385] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9826.678386] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9832.578758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9832.805889] Lustre: lustre-OST0000: new disk, initializing [ 9832.810426] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9832.816513] Lustre: 318758:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9832.902243] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9837.854231] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9842.236623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9842.252143] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9842.348600] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9847.001720] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9852.037811] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9852.261380] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9857.774205] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9858.006613] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9863.136910] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9863.137128] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9863.143862] LustreError: Skipped 1 previous similar message [ 9863.146468] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9863.153141] Lustre: Skipped 1 previous similar message [ 9868.262391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9868.272809] Lustre: Skipped 1 previous similar message [ 9873.380706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9873.397345] Lustre: Skipped 1 previous similar message [ 9874.399537] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9874.783524] Lustre: server umount lustre-OST0000 complete [ 9881.059436] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9881.075992] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9883.618087] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9883.620085] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9883.627732] Lustre: Skipped 1 previous similar message [ 9884.971827] Lustre: server umount lustre-MDT0000 complete [ 9888.419510] LustreError: 316784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787608773 with bad export cookie 17043517889015991397 [ 9888.422833] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9888.436364] LustreError: 316784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9888.840828] Lustre: server umount lustre-MDT0001 complete [ 9892.284666] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9894.319753] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:59:37 (1787608777) [ 9900.781268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9901.294704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9906.389953] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9906.666838] LustreError: 321071:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9906.684096] LustreError: 321071:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9910.758639] LustreError: 321070:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9914.623750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9915.016079] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9919.192633] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9927.664573] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9932.249238] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9938.961326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9939.282068] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9946.646566] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9955.863826] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9964.318602] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9964.600384] Lustre: lustre-OST0001: new disk, initializing [ 9964.610865] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9964.621860] Lustre: 323906:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9964.800653] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9971.740526] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [ 9973.406932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9973.428680] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9973.464054] LustreError: 324340:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9974.500703] LustreError: 324340:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9974.515175] LustreError: 324340:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9975.533218] LustreError: 324347:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9977.640695] LustreError: 324340:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9977.653289] LustreError: 324340:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9981.795659] LustreError: 324347:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9981.802555] LustreError: 324347:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [ 9982.602445] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9982.856906] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9989.184339] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9d85d0184000.ost_server_uuid 50 [ 9989.988745] LustreError: 324347:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9990.001021] LustreError: 324347:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [ 9991.156842] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9d85d0184000.ost_server_uuid in FULL state after 0 sec [ 9996.783343] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9d85d0184000.ost_server_uuid 50 [ 9998.165914] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9d85d0184000.ost_server_uuid in FULL state after 0 sec [10000.380032] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10013.740897] Lustre: lustre-OST0001: Not available for connect from 192.168.206.16@tcp (stopping) [10013.751610] Lustre: Skipped 1 previous similar message [10014.691517] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10019.492293] Lustre: server umount lustre-OST0001 complete [10019.813188] LustreError: 322893:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10019.838198] LustreError: 322893:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10024.941272] LustreError: 322892:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10024.972610] LustreError: 322892:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10027.491205] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10027.501187] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10027.527886] Lustre: Skipped 1 previous similar message [10030.052770] LustreError: 322893:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10030.075407] LustreError: 322893:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10031.909511] Lustre: server umount lustre-OST0000 complete [10037.728346] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10037.736303] LustreError: Skipped 1 previous similar message [10041.784667] Lustre: server umount lustre-MDT0000 complete [10045.412900] LustreError: 322119:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10045.434108] LustreError: 322119:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10045.568377] LustreError: 321052:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787608930 with bad export cookie 17043517889015992286 [10045.574972] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10045.579377] LustreError: 321052:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10046.014990] Lustre: server umount lustre-MDT0001 complete [10055.835130] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [10058.472969] Key type lgssc unregistered [10058.771759] LNet: 326610:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10058.780816] LNetError: 326610:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10058.793893] LNet: Removed LNI 192.168.206.116@tcp [10059.662909] Key type .llcrypt unregistered [10059.666629] Key type ._llcrypt unregistered [10082.479754] Key type ._llcrypt registered [10082.481729] Key type .llcrypt registered [10082.562100] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [10083.559880] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10083.675039] alg: No test for adler32 (adler32-zlib) [10084.736880] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [10085.163776] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [10086.863494] Key type lgssc registered [10088.213388] Lustre: Echo OBD driver; http://www.lustre.org/ [10097.264095] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10097.285837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10098.891232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10104.554172] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10109.418423] LustreError: 328034:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10109.446549] LustreError: 328034:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10113.230914] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10114.531708] LustreError: 328033:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10119.654165] LustreError: 328034:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10120.962313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10121.518770] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10125.654448] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10134.109027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10134.471205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10140.213548] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10144.706492] LustreError: 329713:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10144.727313] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10147.424973] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10150.880952] LustreError: 329713:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10150.910133] LustreError: 329713:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10154.368431] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10154.554270] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10159.621649] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10160.124961] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10167.977192] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10176.001943] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10176.343731] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10282.468244] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10282.470252] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10282.475862] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10282.480130] LustreError: Skipped 1 previous similar message [10286.730234] Lustre: server umount lustre-OST0001 complete [10287.589403] LustreError: 329713:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10287.611247] LustreError: 329713:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10293.216534] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10293.226900] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10293.239113] Lustre: Skipped 1 previous similar message [10293.244851] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10293.249775] Lustre: Skipped 1 previous similar message [10294.240418] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10294.244951] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10294.258593] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10297.845408] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10297.863155] Lustre: Skipped 1 previous similar message [10299.065924] Lustre: server umount lustre-OST0000 complete [10306.020154] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10306.045464] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10306.075965] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10309.772438] Lustre: server umount lustre-MDT0000 complete [10312.965386] LustreError: 328013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787609198 with bad export cookie 11128004599498924181 [10312.967040] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10312.990893] LustreError: 328013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10313.421803] Lustre: server umount lustre-MDT0001 complete [10324.208805] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [10326.953326] Key type lgssc unregistered [10327.308487] LNet: 332977:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10327.323883] LNetError: 332977:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10327.346876] LNet: Removed LNI 192.168.206.116@tcp [10328.372134] Key type .llcrypt unregistered [10328.374747] Key type ._llcrypt unregistered [10366.094790] Key type ._llcrypt registered [10366.099126] Key type .llcrypt registered [10366.217169] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [10367.259291] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10367.386133] alg: No test for adler32 (adler32-zlib) [10368.418962] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [10368.739596] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [10370.415192] Key type lgssc registered [10371.470625] Lustre: Echo OBD driver; http://www.lustre.org/ [10380.411613] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10380.442128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10381.842874] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10385.921416] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10392.035948] LustreError: 335249:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10392.054657] LustreError: 335249:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10395.118593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10395.656412] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10400.062654] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10403.108831] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10410.542617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10410.845496] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10417.407086] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10420.088988] LustreError: 336741:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10425.330140] LustreError: 336739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10426.993205] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10427.272026] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10428.350805] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10428.353839] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10432.702928] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10440.831580] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10452.103566] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10464.235183] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10464.245271] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10464.253953] Lustre: Skipped 2 previous similar messages [10464.270090] Lustre: Skipped 3 previous similar messages [10469.309938] Lustre: server umount lustre-MDT0000 complete [10469.353100] LustreError: 335249:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10469.372252] LustreError: 335249:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10472.969888] LustreError: 335229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787609358 with bad export cookie 6669360422634163757 [10472.979202] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10472.979609] LustreError: 335229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10473.399390] Lustre: server umount lustre-MDT0001 complete [10487.698931] Lustre: server umount lustre-OST0000 complete [10489.823206] Lustre: 334509:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787609359/real 1787609359] req@ffff92674495b480 x1874444161187328/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787609375 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10489.840538] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10489.848612] Lustre: Skipped 1 previous similar message [10491.409812] Lustre: server umount lustre-OST0001 complete [10503.741939] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [10511.651181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10511.884850] LustreError: 340513:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10511.939460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10515.307786] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10522.088791] LustreError: 340513:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10522.109467] LustreError: 340513:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10522.815468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10523.256711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10527.492307] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10530.580805] Lustre: 341654:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10541.141460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10541.935142] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10548.078339] LustreError: 342009:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10548.365705] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10557.166950] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10557.426697] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10563.067911] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10563.435660] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10570.398425] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10573.714780] Lustre: 343523:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10578.407752] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10578.412385] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10578.423690] Lustre: Skipped 3 previous similar messages [10578.436150] Lustre: Skipped 3 previous similar messages [10583.527285] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10583.533057] Lustre: Skipped 3 previous similar messages [10583.896749] Lustre: server umount lustre-MDT0000 complete [10587.459926] LustreError: 340493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787609472 with bad export cookie 6669360422634165465 [10587.460766] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10587.469395] LustreError: 340493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10587.842357] Lustre: server umount lustre-MDT0001 complete [10601.655364] Lustre: server umount lustre-OST0000 complete [10604.002801] Lustre: 334512:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787609473/real 1787609473] req@ffff9267534d0380 x1874444161234432/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787609489 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10604.024869] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10605.352675] Lustre: server umount lustre-OST0001 complete [10608.705800] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10610.437720] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10612.591375] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 18:11:35 (1787609495) [10632.439439] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [10640.452028] loop0: detected capacity change from 0 to 1099511627776 [10640.874746] SGI XFS with ACLs, security attributes, quota, no debug enabled [10640.882192] *** VALIDATE xfs *** [10640.910665] XFS (loop0): Mounting V5 Filesystem [10640.939102] XFS (loop0): Ending clean mount [10641.967246] loop1: detected capacity change from 0 to 18691697672192 [10752.056729] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [10761.262520] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [10795.613791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10795.837333] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10795.865272] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10795.976864] Lustre: lustre-MDT0000: new disk, initializing [10796.071978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10796.087558] Lustre: Skipped 1 previous similar message [10796.110747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10800.823041] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10811.277862] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10811.434740] Lustre: 348658:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10811.517180] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10811.524888] Lustre: Skipped 1 previous similar message [10811.653856] Lustre: lustre-MDT0001: new disk, initializing [10811.831908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10811.844468] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10816.872793] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10826.631410] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10832.793836] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10840.994228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10841.426484] Lustre: lustre-OST0000: new disk, initializing [10841.433670] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10841.441751] Lustre: 350625:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10843.288566] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10843.309587] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10843.501078] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10850.160362] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10860.587557] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10865.271451] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10865.464956] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10870.079688] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10870.355662] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10874.348813] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10874.351453] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10874.370384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10874.388914] LustreError: Skipped 1 previous similar message [10878.508828] Lustre: server umount lustre-OST0000 complete [10883.555575] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10883.569491] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10883.616474] Lustre: Skipped 1 previous similar message [10883.629133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10883.637719] Lustre: Skipped 1 previous similar message [10893.799223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10893.836078] Lustre: Skipped 4 previous similar messages [10896.351809] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10896.551663] Lustre: server umount lustre-MDT0000 complete [10898.912762] LustreError: 351299:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10898.944221] LustreError: 351299:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10900.969961] LustreError: 348651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787609786 with bad export cookie 6669360422634167012 [10900.975740] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10900.978806] LustreError: 348651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10901.546383] Lustre: server umount lustre-MDT0001 complete [10912.394625] XFS (loop0): Unmounting Filesystem [10918.023456] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 18:16:41 (1787609801) [10926.381318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10927.175288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10927.182313] Lustre: Skipped 2 previous similar messages [10932.112902] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10940.260660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10945.469054] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10953.538694] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10960.038849] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10967.581984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10974.966882] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [10982.704386] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10996.704562] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10996.715365] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10996.731100] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10996.742295] Lustre: Skipped 2 previous similar messages [10996.750965] Lustre: Skipped 1 previous similar message [11001.279212] Lustre: server umount lustre-OST0000 complete [11021.279936] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11021.834506] Lustre: server umount lustre-MDT0000 complete [11022.314277] LustreError: 353188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11022.355451] LustreError: 353188:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11027.894729] LustreError: 353167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787609913 with bad export cookie 6669360422634167894 [11027.910764] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11027.913092] LustreError: 353167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11028.660650] Lustre: server umount lustre-MDT0001 complete [11041.626068] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [11045.053363] Key type lgssc unregistered [11045.439651] LNet: 356821:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11045.461172] LNetError: 356821:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11045.485608] LNet: Removed LNI 192.168.206.116@tcp [11046.608157] Key type .llcrypt unregistered [11046.610671] Key type ._llcrypt unregistered [11086.727463] Key type ._llcrypt registered [11086.729410] Key type .llcrypt registered [11086.884904] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [11089.792488] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11089.894639] alg: No test for adler32 (adler32-zlib) [11091.046358] Lustre: Lustre: Build Version: 2.17.57_80_g20f51fc [11091.281905] LNet: Added LNI 192.168.206.116@tcp [8/256/0/180] [11092.967238] Key type lgssc registered [11094.407258] Lustre: Echo OBD driver; http://www.lustre.org/ [11102.937395] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11102.991811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11104.353725] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11108.979104] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11113.960633] LustreError: 358395:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11113.992195] LustreError: 358395:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11117.347317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11117.741269] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11121.960812] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11129.665686] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11135.731408] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11143.298214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11143.603455] LustreError: 360215:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11143.730843] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11151.055789] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11158.442689] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11165.156311] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11165.176735] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11165.184588] Lustre: Skipped 1 previous similar message [11165.449702] Lustre: server umount lustre-MDT0001 complete [11170.274796] LustreError: 358391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11170.307200] LustreError: 358391:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11175.395058] LustreError: 358396:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11175.414076] LustreError: 358396:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11176.338674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11176.472889] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11176.477342] Lustre: Found index 1 for lustre-MDT0001, updating log [11176.525090] Lustre: 358384:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11176.823041] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11181.505481] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11182.084022] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11186.188328] LustreError: 361993:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11186.210626] LustreError: 361993:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11186.221669] LustreError: 358388:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11186.290435] LustreError: 361996:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.116@tcp: cfg command failed: rc = -17 [11186.295972] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11186.295972] [11487.800653] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [11487.810887] Lustre: Skipped 1 previous similar message [11489.257590] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11489.267231] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11489.279112] Lustre: Skipped 1 previous similar message [11489.281446] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11489.300813] Lustre: Skipped 1 previous similar message [11492.500929] Lustre: server umount lustre-MDT0001 complete [11492.889071] LustreError: 358392:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11492.902443] LustreError: 358392:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11503.127527] LustreError: 358392:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11503.139196] LustreError: 358392:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11503.480356] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11503.642342] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11503.649062] Lustre: Found index 1 for lustre-MDT0001, updating log [11503.707737] Lustre: 358385:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11504.036413] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11508.122769] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11509.244818] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11510.788428] LustreError: 370237:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11510.805687] LustreError: 370237:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11510.811372] LustreError: 358388:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11510.816170] LustreError: 358388:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11510.884259] LustreError: 370244:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11510.892245] LustreError: 370244:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11510.900978] LustreError: 370244:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.116@tcp: cfg command failed: rc = -17 [11510.906730] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.116@tcp [11510.906730] [11815.472794] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.16@tcp (stopping) [11816.422579] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11816.428139] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11816.442472] Lustre: Skipped 2 previous similar messages [11820.486965] Lustre: server umount lustre-MDT0001 complete [11820.566972] LustreError: 358392:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11831.744964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11831.936834] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11831.942765] Lustre: Found index 1 for lustre-MDT0001, updating log [11831.990922] Lustre: 358385:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11832.440771] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11836.785348] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [11837.934184] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11838.963746] LustreError: 379065:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11838.976454] LustreError: 379065:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11838.990793] LustreError: 358388:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11838.996145] LustreError: 358388:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11839.123306] LustreError: 379072:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11839.161490] LustreError: 379072:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.116@tcp: cfg command failed: rc = -114 [11839.178399] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11839.178399] [12159.976132] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12159.983485] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12160.002384] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12160.007508] Lustre: Skipped 2 previous similar messages [12162.780719] Lustre: server umount lustre-MDT0000 complete [12165.608058] LustreError: 366421:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12165.631409] LustreError: 366421:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [12166.095327] LustreError: 358378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787611051 with bad export cookie 4302211810833452260 [12166.113154] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12166.117795] LustreError: 358378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12170.721173] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12170.731423] Lustre: Skipped 2 previous similar messages [12170.735842] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12170.743513] Lustre: Skipped 3 previous similar messages [12172.501678] Lustre: server umount lustre-MDT0001 complete [12176.324154] Lustre: server umount lustre-OST0000 complete [12184.488771] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [12192.738242] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [12223.417131] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:38:26 (1787611106) [12228.884705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12229.046226] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12229.064480] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12229.117397] Lustre: lustre-MDT0000: new disk, initializing [12229.162147] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12229.179310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12232.685871] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [12242.306711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12242.386357] Lustre: 390475:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12242.454288] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12242.463568] Lustre: Skipped 1 previous similar message [12242.571111] Lustre: lustre-MDT0001: new disk, initializing [12242.690361] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12242.733688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12242.748266] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12246.547226] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [12253.728754] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12258.132523] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12264.320074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12264.589841] Lustre: lustre-OST0000: new disk, initializing [12264.595180] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12264.600160] Lustre: 392439:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12264.681045] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12269.861169] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_default_debug -1 all [12270.166030] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12270.182678] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12270.283182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12279.167515] Lustre: DEBUG MARKER: oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12285.930402] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12285.934821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12285.944762] Lustre: Skipped 2 previous similar messages [12285.949148] Lustre: Skipped 2 previous similar messages [12291.999397] Lustre: server umount lustre-MDT0000 complete [12295.899427] LustreError: 390467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787611181 with bad export cookie 4302211810833454213 [12295.910591] LustreError: MGC192.168.206.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12295.915221] LustreError: 390467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12295.935042] LustreError: 391279:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12296.165312] LustreError: 390482:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12296.191025] LustreError: 390482:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12296.347698] Lustre: server umount lustre-MDT0001 complete [12302.247982] Lustre: server umount lustre-OST0000 complete [12327.144932] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing set_hostid [12336.646437] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing load_modules_local [12364.540703] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12079 sec ========= 18:40:48 (1787611248) [12366.167249] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:40:49 (1787611249) === [12369.577771] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:40:53 (1787611253) === [12400.406429] Lustre: DEBUG MARKER: oleg616-server.virtnet: executing unload_modules_local [12402.969568] Key type lgssc unregistered [12403.269892] LNet: 398806:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12403.278285] LNetError: 398806:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12403.299632] LNet: Removed LNI 192.168.206.116@tcp [12404.085393] Key type .llcrypt unregistered [12404.089831] Key type ._llcrypt unregistered