[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 515717032 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2480MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002396] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.006983] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008013] pid_max: default: 32768 minimum: 301 [ 0.009124] LSM: Security Framework initializing [ 0.010055] Yama: becoming mindful. [ 0.011038] SELinux: Initializing. [ 0.012065] *** VALIDATE selinux *** [ 0.020653] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024421] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025132] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026106] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028080] *** VALIDATE tmpfs *** [ 0.030013] *** VALIDATE proc *** [ 0.031236] *** VALIDATE cgroup *** [ 0.032009] *** VALIDATE cgroup2 *** [ 0.033265] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034168] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036030] Spectre V2 : User space: Vulnerable [ 0.037008] Speculative Store Bypass: Vulnerable [ 0.040164] debug: unmapping init [mem 0xffffffff97459000-0xffffffff97460fff] [ 0.043164] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044625] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045026] ... version: 2 [ 0.046013] ... bit width: 48 [ 0.047013] ... generic registers: 4 [ 0.048014] ... value mask: 0000ffffffffffff [ 0.049014] ... max period: 00007fffffffffff [ 0.050017] ... fixed-purpose events: 3 [ 0.051013] ... event mask: 000000070000000f [ 0.053229] rcu: Hierarchical SRCU implementation. [ 0.055386] smp: Bringing up secondary CPUs ... [ 0.056617] x86: Booting SMP configuration: [ 0.057027] .... node #0, CPUs: #1 #2 #3 [ 0.060471] smp: Brought up 1 node, 4 CPUs [ 0.062015] smpboot: Max logical packages: 1 [ 0.063022] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.212326] node 0 deferred pages initialised in 147ms [ 0.216021] devtmpfs: initialized [ 0.218386] x86/mm: Memory block size: 128MB [ 0.224272] gcov: version magic: 0x41383552 [ 0.228417] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.229088] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.230438] pinctrl core: initialized pinctrl subsystem [ 0.232352] [ 0.232914] ************************************************************* [ 0.236016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.239012] ** ** [ 0.241012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.244013] ** ** [ 0.247014] ** This means that this kernel is built to expose internal ** [ 0.250016] ** IOMMU data structures, which may compromise security on ** [ 0.253016] ** your system. ** [ 0.256079] ** ** [ 0.259012] ** If you see this message and you are not debugging the ** [ 0.261015] ** kernel, report this immediately to your vendor! ** [ 0.263013] ** ** [ 0.266014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.269014] ************************************************************* [ 0.272207] NET: Registered protocol family 16 [ 0.274502] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.278089] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.281089] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.285018] cpuidle: using governor menu [ 0.286825] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.289797] PCI: Using configuration type 1 for base access [ 0.292333] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.305072] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.307029] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.311074] cryptd: max_cpu_qlen set to 1000 [ 0.313555] ACPI: Added _OSI(Module Device) [ 0.315019] ACPI: Added _OSI(Processor Device) [ 0.317014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.319014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.324360] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.331612] ACPI: Interpreter enabled [ 0.333076] ACPI: PM: (supports S0 S3 S4 S5) [ 0.335016] ACPI: Using IOAPIC for interrupt routing [ 0.337147] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.341427] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.352373] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.355063] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.359025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.362102] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.368494] acpiphp: Slot [2] registered [ 0.370207] acpiphp: Slot [5] registered [ 0.372190] acpiphp: Slot [6] registered [ 0.373180] acpiphp: Slot [7] registered [ 0.375180] acpiphp: Slot [8] registered [ 0.377207] acpiphp: Slot [9] registered [ 0.379165] acpiphp: Slot [10] registered [ 0.381195] acpiphp: Slot [3] registered [ 0.383161] acpiphp: Slot [4] registered [ 0.385144] acpiphp: Slot [11] registered [ 0.387145] acpiphp: Slot [12] registered [ 0.389113] acpiphp: Slot [13] registered [ 0.391124] acpiphp: Slot [14] registered [ 0.392214] acpiphp: Slot [15] registered [ 0.394136] acpiphp: Slot [16] registered [ 0.396222] acpiphp: Slot [17] registered [ 0.398137] acpiphp: Slot [18] registered [ 0.400108] acpiphp: Slot [19] registered [ 0.401123] acpiphp: Slot [20] registered [ 0.403187] acpiphp: Slot [21] registered [ 0.405114] acpiphp: Slot [22] registered [ 0.406107] acpiphp: Slot [23] registered [ 0.408140] acpiphp: Slot [24] registered [ 0.410097] acpiphp: Slot [25] registered [ 0.411268] acpiphp: Slot [26] registered [ 0.413123] acpiphp: Slot [27] registered [ 0.415147] acpiphp: Slot [28] registered [ 0.417123] acpiphp: Slot [29] registered [ 0.419100] acpiphp: Slot [30] registered [ 0.420110] acpiphp: Slot [31] registered [ 0.422160] PCI host bridge to bus 0000:00 [ 0.424025] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.426030] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.429029] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.431044] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.435033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.438032] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.440206] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.443075] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.446440] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.457825] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.462983] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.466027] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.468020] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.472027] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.475744] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.478864] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.482065] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.486013] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.491020] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.505027] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.511941] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.517335] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.529020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.539026] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.561021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.572170] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.586021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.593021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.615021] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.627028] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.637020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.647021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.667024] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.677758] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.688021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.699031] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.731022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.743233] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.753018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.760021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.786022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.796477] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.807022] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.814021] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.842023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.855765] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.858446] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.860511] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.865534] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.868243] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.874043] iommu: Default domain type: Passthrough [ 0.876645] SCSI subsystem initialized [ 0.878199] ACPI: bus type USB registered [ 0.880134] usbcore: registered new interface driver usbfs [ 0.882101] usbcore: registered new interface driver hub [ 0.883096] usbcore: registered new device driver usb [ 0.885262] pps_core: LinuxPPS API ver. 1 registered [ 0.887015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.890079] PTP clock support registered [ 0.892177] EDAC MC: Ver: 3.0.0 [ 0.894176] PCI: Using ACPI for IRQ routing [ 0.896399] NetLabel: Initializing [ 0.898016] NetLabel: domain hash size = 128 [ 0.901017] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.903099] NetLabel: unlabeled traffic allowed by default [ 0.905148] vgaarb: loaded [ 0.906355] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.908015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.916146] clocksource: Switched to clocksource kvm-clock [ 1.029213] VFS: Disk quotas dquot_6.6.0 [ 1.031325] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.034141] *** VALIDATE ramfs *** [ 1.035353] *** VALIDATE hugetlbfs *** [ 1.037186] pnp: PnP ACPI init [ 1.040121] pnp: PnP ACPI: found 6 devices [ 1.057381] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.061395] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.063669] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.066147] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.068666] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.071331] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.074452] NET: Registered protocol family 2 [ 1.077144] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.082505] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.086425] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.092051] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.095616] TCP: Hash tables configured (established 65536 bind 65536) [ 1.098779] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.102067] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.105205] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.108419] NET: Registered protocol family 1 [ 1.112547] RPC: Registered named UNIX socket transport module. [ 1.114522] RPC: Registered udp transport module. [ 1.116321] RPC: Registered tcp transport module. [ 1.118190] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.120597] NET: Registered protocol family 44 [ 1.122189] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.124269] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.126141] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.128370] PCI: CLS 0 bytes, default 64 [ 1.130118] Unpacking initramfs... [ 2.553221] debug: unmapping init [mem 0xffff97a7fcc54000-0xffff97a7fffbffff] [ 2.557812] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.560246] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.563142] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.076167] Initialise system trusted keyrings [ 3.077942] Key type blacklist registered [ 3.079696] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.088874] zbud: loaded [ 3.092428] *** VALIDATE nfs *** [ 3.093950] *** VALIDATE nfs4 *** [ 3.095961] pstore: using deflate compression [ 3.100026] Platform Keyring initialized [ 3.210866] NET: Registered protocol family 38 [ 3.212331] Key type asymmetric registered [ 3.213759] Asymmetric key parser 'x509' registered [ 3.215425] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.218201] io scheduler mq-deadline registered [ 3.220122] io scheduler kyber registered [ 3.222065] io scheduler bfq registered [ 3.224323] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.227741] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.231388] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.234642] ACPI: Power Button [PWRF] [ 3.240736] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.248529] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.264922] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.273985] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.298834] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.329984] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.361982] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.367422] Non-volatile memory driver v1.3 [ 3.369130] Linux agpgart interface v0.103 [ 3.409442] virtio_blk virtio1: [vda] 149768 512-byte logical blocks (76.7 MB/73.1 MiB) [ 3.413494] vda: detected capacity change from 0 to 76681216 [ 3.429293] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.432840] vdb: detected capacity change from 0 to 1073741824 [ 3.448806] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.451136] vdc: detected capacity change from 0 to 2621440000 [ 3.467118] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.470623] vdd: detected capacity change from 0 to 2621440000 [ 3.491032] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.494089] vde: detected capacity change from 0 to 4294967296 [ 3.514929] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.518481] vdf: detected capacity change from 0 to 4294967296 [ 3.526734] libphy: Fixed MDIO Bus: probed [ 3.532248] usbcore: registered new interface driver usbserial_generic [ 3.535274] usbserial: USB Serial support registered for generic [ 3.538065] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.542807] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.545244] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.548139] mousedev: PS/2 mouse device common for all mice [ 3.551677] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.552955] rtc_cmos 00:05: RTC can wake from S4 [ 3.559412] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.562119] rtc_cmos 00:05: registered as rtc0 [ 3.566436] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.566827] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.571266] intel_pstate: CPU model not supported [ 3.579538] hid: raw HID events driver (C) Jiri Kosina [ 3.582829] usbcore: registered new interface driver usbhid [ 3.586327] usbhid: USB HID core driver [ 3.588898] drop_monitor: Initializing network drop monitor service [ 3.591648] Initializing XFRM netlink socket [ 3.593798] NET: Registered protocol family 10 [ 3.597618] Segment Routing with IPv6 [ 3.598855] NET: Registered protocol family 17 [ 3.601896] mpls_gso: MPLS GSO support [ 3.607170] RAS: Correctable Errors collector initialized. [ 3.609620] AVX version of gcm_enc/dec engaged. [ 3.611454] AES CTR mode by8 optimization enabled [ 3.697953] sched_clock: Marking stable (3697927428, 0)->(4669369120, -971441692) [ 3.702651] registered taskstats version 1 [ 3.705317] Loading compiled-in X.509 certificates [ 3.707848] zswap: loaded using pool lzo/zbud [ 3.737753] Key type big_key registered [ 3.752459] Key type encrypted registered [ 3.754886] ima: No TPM chip found, activating TPM-bypass! [ 3.758748] ima: Allocated hash algorithm: sha1 [ 3.761621] ima: No architecture policies found [ 3.764616] evm: Initialising EVM extended attributes: [ 3.767532] evm: security.selinux [ 3.769882] evm: security.ima [ 3.771506] evm: security.capability [ 3.773473] evm: HMAC attrs: 0x1 [ 3.776070] rtc_cmos 00:05: setting system clock to 2026-09-05 19:27:39 UTC (1788636459) [ 3.783451] debug: unmapping init [mem 0xffffffff98403000-0xffffffff985fffff] [ 3.788028] debug: unmapping init [mem 0xffffffff97182000-0xffffffff97458fff] [ 3.798092] Write protecting the kernel read-only data: 28672k [ 3.801947] debug: unmapping init [mem 0xffffffff95803000-0xffffffff959fffff] [ 3.805395] debug: unmapping init [mem 0xffffffff96114000-0xffffffff961fffff] [ 3.843414] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.853530] systemd[1]: Detected virtualization kvm. [ 3.855615] systemd[1]: Detected architecture x86-64. [ 3.857960] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.887668] systemd[1]: No hostname configured. [ 3.889701] systemd[1]: Set hostname to . [ 3.892365] random: systemd: uninitialized urandom read (16 bytes read) [ 3.895316] systemd[1]: Initializing machine ID from random generator. [ 3.923893] random: ln: uninitialized urandom read (6 bytes read) [ 4.028280] random: systemd: uninitialized urandom read (16 bytes read) [ 4.031868] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.039059] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.045684] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.685840] device-mapper: uevent: version 1.0.3 [ 4.688256] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [[ 5.056241] random: fast init done  OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.621620] virtio_net virtio0 ens2: renamed from eth0 [ 5.705916] scsi host0: ata_piix [ 5.756514] scsi host1: ata_piix [ 5.759802] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.762731] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.215185] dracut-initqueue[591]: RTNETLINK answers: File exists [ 10.278844] random: crng init done [ 10.281164] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.881919] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.122635] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.386574] SELinux: Disabled at runtime. [ 12.452042] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.459989] systemd[1]: Detected virtualization kvm. [ 12.461830] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.999743] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.003086] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.008466] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.012955] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.016723] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.027213] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.034975] systemd[1]: Mounting POSIX Message Queue File System... Mounting POSIX Message Queue File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-getty.slice. Activating swap /dev/disk/by-label/SWAP... Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 13.084582] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice User and Session Slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on udev Kernel Socket. Mounting Kernel Debug File System... [ OK ] Reached target Slices. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting Huge Pages File System... [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Paths. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.497747] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.871703] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.890180] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.066404] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.085698] EDAC sbridge: Ver: 1.1.2 [ 15.489443] Key type dns_resolver registered [ 15.828465] NFS: Registering the id_resolver key type [ 15.830471] Key type id_resolver registered [ 15.831868] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Login Service... Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg316-server login: [ 64.017296] libcfs: loading out-of-tree module taints kernel. [ 64.054571] Key type ._llcrypt registered [ 64.057294] Key type .llcrypt registered [ 64.153618] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 81.112214] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 82.498690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 82.519385] alg: No test for adler32 (adler32-zlib) [ 84.039981] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 84.832968] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 86.608191] Key type lgssc registered [ 88.520256] Lustre: Echo OBD driver; http://www.lustre.org/ [ 90.045058] hrtimer: interrupt took 4048609 ns [ 105.808673] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 148.756589] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 160.763223] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 160.796964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 162.056143] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 162.081240] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 162.169846] Lustre: lustre-MDT0000: new disk, initializing [ 162.255937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 162.281335] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 166.624304] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 179.247835] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 179.392887] Lustre: 6514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 179.427580] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 179.432209] Lustre: Skipped 1 previous similar message [ 179.525484] Lustre: lustre-MDT0001: new disk, initializing [ 179.608476] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 179.633121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 179.659101] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 184.319240] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 189.087759] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 199.128138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 199.408208] Lustre: lustre-OST0000: new disk, initializing [ 199.411267] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 199.418244] Lustre: 8452:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 199.479173] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 206.260673] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.913688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 207.937544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 208.036064] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 223.859684] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 224.072232] Lustre: lustre-OST0001: new disk, initializing [ 224.081177] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 224.087923] Lustre: 9526:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 224.172922] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 233.193986] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.654774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 233.670540] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 233.786671] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 246.848581] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 253.025573] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 258.280516] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing check_logdir /tmp/testlogs/ [ 262.762194] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing yml_node [ 268.660817] Lustre: DEBUG MARKER: Client: 2.17.58.2 [ 271.015703] Lustre: DEBUG MARKER: MDS: 2.17.58.2 [ 273.171816] Lustre: DEBUG MARKER: OSS: 2.17.58.2 [ 274.621554] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Sep 5 15:32:07 EDT 2026 [ 290.651545] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 292.232213] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 310.757108] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 310.770560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 310.803864] Lustre: Skipped 3 previous similar messages [ 310.812636] Lustre: Skipped 2 previous similar messages [ 314.174438] Lustre: server umount lustre-MDT0000 complete [ 317.608940] LustreError: 6506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788636773 with bad export cookie 376274308286325139 [ 317.610315] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 317.626830] LustreError: 6506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 320.994754] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 320.996536] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 321.016965] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 321.024550] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 321.033812] Lustre: Skipped 2 previous similar messages [ 323.049876] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 323.049988] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 323.078614] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 323.088912] Lustre: Skipped 1 previous similar message [ 324.080676] Lustre: server umount lustre-MDT0001 complete [ 328.348981] Lustre: server umount lustre-OST0000 complete [ 331.825596] Lustre: server umount lustre-OST0001 complete [ 337.648507] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 346.352623] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 383.018106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 383.234658] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 383.258518] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 383.347383] Lustre: lustre-MDT0000: new disk, initializing [ 383.470911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 383.494950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 388.054478] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 398.360673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 398.556778] Lustre: 15082:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 398.628313] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 398.635618] Lustre: Skipped 1 previous similar message [ 398.709713] Lustre: lustre-MDT0001: new disk, initializing [ 398.832439] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 398.884451] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 398.896116] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 403.674969] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 413.204992] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 419.252972] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 425.100851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 425.377087] Lustre: lustre-OST0000: new disk, initializing [ 425.382966] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 425.390532] Lustre: 17047:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 425.499368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 426.706522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 426.718903] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 426.839587] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 430.980864] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 439.209534] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 443.828554] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 444.012444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 448.484941] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 448.672495] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 452.579722] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 452.582758] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 452.599253] Lustre: Skipped 2 previous similar messages [ 452.605125] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 452.608805] Lustre: Skipped 1 previous similar message [ 456.532685] Lustre: server umount lustre-OST0000 complete [ 460.257501] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 460.263965] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 460.287163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 466.458059] Lustre: server umount lustre-MDT0000 complete [ 467.939562] LustreError: 15093:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 467.950539] LustreError: 15093:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 470.517820] LustreError: 17046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788636926 with bad export cookie 376274308286326665 [ 470.525756] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 470.530893] LustreError: 17046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 471.115671] Lustre: server umount lustre-MDT0001 complete [ 476.461698] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 15:35:29 (1788636929) [ 482.636511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 483.055381] LustreError: 19315:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 483.201398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 487.697689] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 488.430132] LustreError: 19316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 495.957035] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 500.515389] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 508.267712] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 513.752923] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 521.183180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 527.484819] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 535.329503] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 542.394696] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 551.406657] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 551.417038] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 551.420148] Lustre: Skipped 3 previous similar messages [ 551.426837] Lustre: Skipped 3 previous similar messages [ 554.874356] Lustre: server umount lustre-MDT0000 complete [ 556.519969] LustreError: 19312:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 556.567702] LustreError: 19312:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 558.586745] LustreError: 19296:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788637014 with bad export cookie 376274308286327554 [ 558.602943] LustreError: 19296:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 558.608075] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 559.105962] Lustre: server umount lustre-MDT0001 complete [ 573.344714] Lustre: server umount lustre-OST0000 complete [ 591.921355] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 600.541425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 600.914270] LustreError: 23696:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.965306] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 600.975319] Lustre: Skipped 2 previous similar messages [ 604.663599] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 612.223978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 616.453691] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 618.952888] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 624.347082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 629.583339] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 637.029661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 637.176251] Lustre: lustre-OST0001: new disk, initializing [ 637.179905] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 637.184896] Lustre: 26010:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 638.972902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 638.992173] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 639.101482] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 642.644156] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 651.509891] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 654.707258] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 664.548400] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 664.553605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 664.583514] Lustre: Skipped 1 previous similar message [ 664.587199] Lustre: Skipped 1 previous similar message [ 669.153620] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 670.392145] Lustre: server umount lustre-MDT0000 complete [ 673.251214] LustreError: 26009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788637128 with bad export cookie 376274308286328394 [ 673.251567] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 673.262051] LustreError: 26009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 673.486499] Lustre: server umount lustre-MDT0001 complete [ 687.144424] Lustre: server umount lustre-OST0000 complete [ 700.921595] Lustre: server umount lustre-OST0001 complete [ 716.169964] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 725.672542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 726.268217] LustreError: 29037:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 726.289655] LustreError: 29037:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 730.383768] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 736.924101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 737.239865] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 737.247697] Lustre: Skipped 4 previous similar messages [ 741.003256] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 743.960073] Lustre: 30185:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 749.689929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 755.698809] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 764.644066] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 773.381790] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 780.900920] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 785.664532] Lustre: 32062:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 793.570737] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 793.584157] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 793.603185] Lustre: Skipped 2 previous similar messages [ 793.614045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 793.620172] Lustre: Skipped 6 previous similar messages [ 797.526798] Lustre: server umount lustre-MDT0000 complete [ 801.009682] LustreError: 29018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788637256 with bad export cookie 376274308286329689 [ 801.019494] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 801.034813] LustreError: 29018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 801.276392] Lustre: server umount lustre-MDT0001 complete [ 816.608680] Lustre: 3649:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788637256/real 1788637256] req@ffff97a8477c0a80 x1875521348851840/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788637272 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 816.790280] Lustre: server umount lustre-OST0000 complete [ 820.988776] Lustre: server umount lustre-OST0001 complete [ 830.383735] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 15:41:23 (1788637283) [ 857.972566] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 867.416303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 867.797922] LustreError: 34864:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 867.818973] LustreError: 34864:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 871.958374] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 879.494553] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 883.536405] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 885.998886] Lustre: 36012:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 891.887092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 897.557569] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 905.171681] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 912.402372] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 919.604407] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 922.675351] Lustre: 37894:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 931.302832] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 931.314468] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 931.315207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 931.315213] Lustre: Skipped 4 previous similar messages [ 931.318524] Lustre: Skipped 5 previous similar messages [ 933.018417] Lustre: server umount lustre-MDT0000 complete [ 935.741293] LustreError: 36014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788637391 with bad export cookie 376274308286330802 [ 935.751109] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 935.752598] LustreError: 36014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 935.998134] Lustre: server umount lustre-MDT0001 complete [ 949.232459] Lustre: server umount lustre-OST0000 complete [ 961.612319] Lustre: server umount lustre-OST0001 complete [ 969.578301] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 15:43:42 (1788637422) [ 983.583949] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 992.656664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 996.700799] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 998.370922] LustreError: 40037:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 998.401903] LustreError: 40037:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 1004.249622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1004.592726] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1004.600512] Lustre: Skipped 7 previous similar messages [ 1008.426943] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1011.135554] Lustre: 41187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1017.509100] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1023.838245] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1031.386131] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1037.084231] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1044.116240] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1047.158885] Lustre: 43065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1048.082524] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1048.086528] Lustre: Skipped 1 previous similar message [ 1052.133145] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1052.158280] Lustre: Skipped 5 previous similar messages [ 1057.195871] Lustre: server umount lustre-MDT0000 complete [ 1060.719684] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788637516 with bad export cookie 376274308286331901 [ 1060.727312] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1060.737246] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1062.372915] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 1062.382641] Lustre: Skipped 9 previous similar messages [ 1075.169190] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1075.420247] Lustre: server umount lustre-MDT0001 complete [ 1082.930190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1085.922543] LustreError: 3645:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff97a743b93b80 x1875521348949376/t0(0) o250->MGC192.168.203.116@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1090.684190] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1098.280595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1100.783354] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1103.400947] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1111.640116] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1113.648449] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1119.362168] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1120.698878] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1132.061096] Lustre: server umount lustre-MDT0000 complete [ 1135.576970] LustreError: 40016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788637591 with bad export cookie 376274308286333007 [ 1135.583141] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1135.599507] LustreError: 40016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1135.910184] Lustre: server umount lustre-MDT0001 complete [ 1149.864352] Lustre: server umount lustre-OST0000 complete [ 1163.349615] Lustre: server umount lustre-OST0001 complete [ 1171.634768] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 15:47:04 (1788637624) [ 1188.310651] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1197.824306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1202.793595] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1210.941908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1215.933576] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1219.150713] Lustre: 48823:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1224.691720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1230.734617] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1239.076765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1244.243447] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1251.065563] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1254.830855] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1267.690819] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1267.702042] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1267.715617] Lustre: Skipped 6 previous similar messages [ 1271.210919] Lustre: server umount lustre-MDT0000 complete [ 1274.341444] LustreError: 47670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1274.366988] LustreError: 47670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 31 previous similar messages [ 1274.588865] LustreError: 48826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788637730 with bad export cookie 376274308286333826 [ 1274.590864] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1274.596222] LustreError: 48826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1274.910708] Lustre: server umount lustre-MDT0001 complete [ 1287.894284] Lustre: server umount lustre-OST0000 complete [ 1301.504410] Lustre: server umount lustre-OST0001 complete [ 1311.529597] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 15:49:24 (1788637764) [ 1314.111322] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1316.122492] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 15:49:28 (1788637768) [ 1327.307116] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 1334.811988] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1355.675450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1355.916605] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1356.023441] Lustre: lustre-MDT0000: new disk, initializing [ 1356.115385] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1360.382436] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1370.998574] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1378.935295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1379.472745] Lustre: lustre-OST0000: new disk, initializing [ 1379.480836] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1379.489472] Lustre: Skipped 1 previous similar message [ 1379.498987] Lustre: 54901:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1381.003425] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1381.013837] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1381.056067] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1386.853257] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1397.250742] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1399.576948] Lustre: DEBUG MARKER: create test files [ 1437.160055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1437.161262] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1437.171082] Lustre: Skipped 19 previous similar messages [ 1441.329514] Lustre: server umount lustre-OST0000 complete [ 1445.407839] Lustre: server umount lustre-MDT0000 complete [ 1457.256863] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1460.000520] Key type lgssc unregistered [ 1460.257803] LNet: 56530:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1460.270694] LNetError: 56530:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1460.288604] LNet: Removed LNI 192.168.203.116@tcp [ 1461.373146] Key type .llcrypt unregistered [ 1461.374959] Key type ._llcrypt unregistered [ 1491.805312] Key type ._llcrypt registered [ 1491.809561] Key type .llcrypt registered [ 1491.943819] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1493.469674] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1493.533417] alg: No test for adler32 (adler32-zlib) [ 1495.089835] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 1495.592788] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1497.432645] Key type lgssc registered [ 1499.205429] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1509.588609] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1509.611964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1511.236591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1516.235306] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1525.407895] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1532.738382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1533.125575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1536.190373] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1536.275253] Lustre: 59262:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1536.286679] Lustre: 59262:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1536.295344] Lustre: 59262:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1536.303207] Lustre: 59262:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1536.309608] Lustre: 59262:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1536.326333] Lustre: 59262:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1539.367826] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1546.396640] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1551.511051] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1557.356205] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1565.594457] Lustre: server umount lustre-OST0000 complete [ 1579.801908] Lustre: server umount lustre-MDT0000 complete [ 1593.500654] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1597.264484] Key type lgssc unregistered [ 1597.565862] LNet: 60876:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1597.574667] LNetError: 60876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1597.597355] LNet: Removed LNI 192.168.203.116@tcp [ 1598.543367] Key type .llcrypt unregistered [ 1598.545342] Key type ._llcrypt unregistered [ 1630.889893] Key type ._llcrypt registered [ 1630.896194] Key type .llcrypt registered [ 1631.013303] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1632.196750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1632.218617] alg: No test for adler32 (adler32-zlib) [ 1633.290895] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 1633.551645] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1635.288216] Key type lgssc registered [ 1636.604945] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1645.015707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1645.027665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1646.480544] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1650.962227] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1658.634630] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1664.920452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1665.220976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1670.691747] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1671.359251] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1671.434983] Lustre: 63713:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1671.441131] Lustre: 63713:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1671.446224] Lustre: 63713:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1671.451042] Lustre: 63713:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1671.458209] Lustre: 63713:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1671.464155] Lustre: 63713:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1677.506181] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1683.194381] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1697.255153] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1697.262254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1697.277129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1698.534538] Lustre: server umount lustre-OST0000 complete [ 1702.507534] Lustre: server umount lustre-MDT0000 complete [ 1713.787702] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1717.703766] Key type lgssc unregistered [ 1718.139119] LNet: 65269:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1718.168553] LNetError: 65269:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1718.202865] LNet: Removed LNI 192.168.203.116@tcp [ 1719.464181] Key type .llcrypt unregistered [ 1719.466779] Key type ._llcrypt unregistered [ 1742.068838] Key type ._llcrypt registered [ 1742.074749] Key type .llcrypt registered [ 1742.149945] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 1758.116368] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1758.979127] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1758.996813] alg: No test for adler32 (adler32-zlib) [ 1760.176696] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 1760.427563] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1762.122453] Key type lgssc registered [ 1763.126493] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1802.522974] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1802.566876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1803.948774] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1803.982193] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1804.154623] Lustre: lustre-MDT0000: new disk, initializing [ 1804.248682] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1804.272220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1810.181531] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1822.716942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1822.896928] Lustre: 68416:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1822.987070] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1822.991456] Lustre: Skipped 1 previous similar message [ 1823.139159] Lustre: lustre-MDT0001: new disk, initializing [ 1823.271652] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1823.361870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1823.382947] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1828.149443] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1838.441881] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1845.181972] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1852.721873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1853.045171] Lustre: lustre-OST0000: new disk, initializing [ 1853.048414] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1853.064359] Lustre: 70387:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1853.186613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1858.672406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1858.685906] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1858.822022] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1861.074375] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1872.868587] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1878.278824] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1878.567668] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1884.847722] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1885.123843] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1889.761958] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1889.770989] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1889.773353] LustreError: Skipped 1 previous similar message [ 1889.775541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1889.818325] Lustre: Skipped 1 previous similar message [ 1893.317574] Lustre: server umount lustre-OST0000 complete [ 1900.006549] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1900.018115] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1900.030381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1900.038694] Lustre: Skipped 1 previous similar message [ 1900.042967] Lustre: Skipped 1 previous similar message [ 1903.076978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1903.086870] Lustre: Skipped 1 previous similar message [ 1908.199308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1912.288159] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1912.568686] Lustre: server umount lustre-MDT0000 complete [ 1916.182593] LustreError: 68408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788638371 with bad export cookie 5708948535881927288 [ 1916.185921] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1916.195876] LustreError: 68408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1916.544434] Lustre: server umount lustre-MDT0001 complete [ 1924.345967] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 15:59:37 (1788638377) [ 1933.878891] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1979.325311] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 1992.521766] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2035.084349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2035.206903] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2035.228610] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2035.288751] Lustre: lustre-MDT0000: new disk, initializing [ 2035.365474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2035.383962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2040.406741] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2052.771470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2052.925726] Lustre: 76426:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2053.016907] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2053.026123] Lustre: Skipped 1 previous similar message [ 2053.148590] Lustre: lustre-MDT0001: new disk, initializing [ 2053.255584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2053.317687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2053.347650] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2057.974620] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2068.245562] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2073.656840] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2080.090156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2080.316757] Lustre: lustre-OST0000: new disk, initializing [ 2080.321612] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2080.340599] Lustre: 78396:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2080.462926] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2086.971143] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2090.540439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2090.553974] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2090.678848] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2097.203555] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2102.171759] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2102.451670] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2106.821402] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2107.017786] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2111.461504] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2111.465149] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2111.477554] LustreError: Skipped 1 previous similar message [ 2111.480951] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2111.506531] Lustre: Skipped 1 previous similar message [ 2111.525856] Lustre: Skipped 4 previous similar messages [ 2121.697980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2121.716289] Lustre: Skipped 3 previous similar messages [ 2123.232146] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2123.403213] Lustre: server umount lustre-OST0000 complete [ 2130.404652] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2130.412640] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2133.714558] Lustre: server umount lustre-MDT0000 complete [ 2137.060805] LustreError: 76438:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2137.079518] LustreError: 76438:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2137.539558] LustreError: 76418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788638593 with bad export cookie 5708948535881928170 [ 2137.547201] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2137.553303] LustreError: 76418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2138.124473] Lustre: server umount lustre-MDT0001 complete [ 2147.641151] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:03:20 (1788638600) [ 2153.885180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2154.482292] LustreError: 80809:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2154.562494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2159.585245] LustreError: 80810:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2159.670110] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2164.707185] LustreError: 80809:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2167.978112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2172.744285] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2180.021288] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2186.234595] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2193.102651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2193.669246] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2193.678108] Lustre: Skipped 1 previous similar message [ 2200.590799] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2209.238582] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2213.573831] Lustre: 83479:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 666cc8dd-6f01-4e53-8a84-45bb819c732f at adminstrative request [ 2215.394525] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2215.427235] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2225.633504] Lustre: MGS: Client 666cc8dd-6f01-4e53-8a84-45bb819c732f (at 0@lo) reconnecting [ 2225.651783] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x4f3a4272323a915c to 0x4f3a4272323a93fc [ 2225.675272] Lustre: MGC192.168.203.116@tcp: Connection restored to 0@lo (at 0@lo) [ 2252.405443] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2252.558510] Lustre: lustre-OST0001: new disk, initializing [ 2252.562798] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2252.569367] Lustre: 83876:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2259.102233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2259.116212] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2259.251577] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2260.573131] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2270.130786] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2274.797752] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2274.800043] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2274.806759] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2274.806768] Lustre: Skipped 3 previous similar messages [ 2274.818097] Lustre: Skipped 2 previous similar messages [ 2274.847701] LustreError: Skipped 1 previous similar message [ 2279.162366] Lustre: server umount lustre-OST0001 complete [ 2279.913076] LustreError: 82632:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2279.929405] LustreError: 82632:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2285.027222] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2289.387265] Lustre: server umount lustre-OST0000 complete [ 2300.101949] Lustre: server umount lustre-MDT0000 complete [ 2300.386361] LustreError: 80809:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2300.411434] LustreError: 80809:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2303.987346] LustreError: 80790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788638759 with bad export cookie 5708948535881929724 [ 2303.992171] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2303.997733] LustreError: 80790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2304.331953] Lustre: server umount lustre-MDT0001 complete [ 2313.726374] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:06:06 (1788638766) [ 2315.347514] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2317.352895] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:06:10 (1788638770) [ 2319.029968] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2320.974492] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:06:13 (1788638773) [ 2322.652619] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2324.692175] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:06:17 (1788638777) [ 2326.475499] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2328.550158] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:06:21 (1788638781) [ 2330.195575] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2336.198321] loop: module loaded [ 2336.224092] loop0: detected capacity change from 0 to 4294967296 [ 2336.315530] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2337.809296] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2341.120140] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2345.296666] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2371.619746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2371.675990] LustreError: 87999:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2371.688819] LustreError: 87999:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2371.700985] LustreError: 87999:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2371.709734] LustreError: 87999:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2392.379056] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 2403.288100] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2444.320444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2444.629996] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2444.656156] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2444.756473] Lustre: lustre-MDT0000: new disk, initializing [ 2444.883714] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2444.893047] Lustre: Skipped 1 previous similar message [ 2444.919752] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2450.237966] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2462.184748] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2462.363236] Lustre: 90865:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2462.695841] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2467.777861] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2478.441098] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2485.577819] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2495.057993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2495.750507] Lustre: 92835:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2503.207914] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2503.367706] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2504.732625] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2515.353293] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2521.009112] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2521.283447] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2526.387957] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2526.599552] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2529.251451] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2529.251882] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2529.263317] LustreError: Skipped 1 previous similar message [ 2529.264407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2529.264603] Lustre: Skipped 5 previous similar messages [ 2529.304988] Lustre: Skipped 4 previous similar messages [ 2534.853238] Lustre: server umount lustre-OST0000 complete [ 2553.312201] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2553.621689] Lustre: server umount lustre-MDT0000 complete [ 2554.863101] LustreError: 90878:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2554.895348] LustreError: 90878:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2557.732379] LustreError: 91669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639013 with bad export cookie 5708948535881930340 [ 2557.735534] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2557.748687] LustreError: 91669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2558.210632] Lustre: server umount lustre-MDT0001 complete [ 2567.708271] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:10:20 (1788639020) [ 2574.610229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2575.341355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2575.349935] Lustre: Skipped 2 previous similar messages [ 2580.356069] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2590.285137] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2595.811094] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2604.027071] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2610.292948] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2618.421042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2625.899578] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2633.996335] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2638.837055] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2639.068891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2643.730080] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2644.028156] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2651.498515] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2651.686258] Lustre: lustre-OST0001: new disk, initializing [ 2651.690062] Lustre: Skipped 2 previous similar messages [ 2651.693576] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2651.697564] Lustre: Skipped 3 previous similar messages [ 2651.701973] Lustre: 98621:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2659.021949] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2659.921321] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2659.936192] Lustre: Skipped 2 previous similar messages [ 2659.943773] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2660.097495] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2670.311600] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2676.395080] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2676.581313] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2681.431792] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2681.618802] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2702.552438] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2729.293508] Lustre: Failing over lustre-MDT0000 [ 2729.716571] Lustre: server umount lustre-MDT0000 complete [ 2730.154584] LustreError: 95248:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2730.194491] LustreError: 95248:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2732.007578] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2732.026158] Lustre: Skipped 4 previous similar messages [ 2747.361402] Lustre: 66639:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788639187/real 1788639187] req@00000000794734ec x1875523106265600/t0(0) o400->MGC192.168.203.116@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788639203 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2747.395519] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2753.943636] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2753.950083] LDISKFS-fs (dm-0): recovery complete [ 2753.961173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2757.622388] LustreError: 66638:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000d3927302 x1875523106274944/t0(0) o250->MGC192.168.203.116@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2758.132342] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2760.839531] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2760.839734] Lustre: 95247:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2763.101686] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2763.241568] Lustre: 95251:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2763.252497] Lustre: 95251:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2763.267930] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2763.379042] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2763.720678] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2764.057443] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2764.072240] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2764.401219] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2764.414570] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2764.445869] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2764.456655] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2765.109859] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2765.132464] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2765.457317] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2765.467000] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2766.766361] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2766.784555] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2767.116535] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2767.138481] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2767.472134] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2767.478304] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2771.017579] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2771.048252] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2771.400465] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2771.426516] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2771.753674] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2771.764245] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2779.158424] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2779.191124] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2779.527594] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2779.549784] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2779.880173] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2779.893045] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2795.251180] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2795.262054] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 49 previous similar messages [ 2795.595669] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2795.610633] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2795.952134] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2795.960548] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2820.216204] Lustre: 101472:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2820.235075] Lustre: 101472:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2820.242151] Lustre: 101472:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 547b9be3-185e-46da-a16c-a47cc7b3e664@192.168.203.16@tcp [ 2820.253842] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2820.257763] Lustre: 101472:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000f5fd302a x1875523088224896/t0(8589934773) o101->547b9be3-185e-46da-a16c-a47cc7b3e664@192.168.203.16@tcp:496/0 lens 592/0 e 0 to 0 dl 1788639286 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2820.309508] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2820.333936] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2820.348630] Lustre: lustre-MDT0000: Denying connection for new client 547b9be3-185e-46da-a16c-a47cc7b3e664 (at 192.168.203.16@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2820.417270] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2820.431856] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2820.436983] Lustre: Skipped 2 previous similar messages [ 2820.525578] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2820.529468] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2840.040042] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2840.053811] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2840.065271] Lustre: Skipped 2 previous similar messages [ 2840.075744] Lustre: Skipped 10 previous similar messages [ 2852.320186] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2852.510218] Lustre: server umount lustre-OST0000 complete [ 2855.408765] LustreError: 97205:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2855.455946] LustreError: 97205:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 44 previous similar messages [ 2860.003190] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2860.012235] LustreError: Skipped 1 previous similar message [ 2872.289836] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2872.629622] Lustre: server umount lustre-OST0001 complete [ 2883.167752] Lustre: server umount lustre-MDT0000 complete [ 2886.951398] LustreError: 95231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639342 with bad export cookie 5708948535881981559 [ 2886.963143] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2886.965531] LustreError: 95231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2887.471415] Lustre: server umount lustre-MDT0001 complete [ 2897.249261] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:15:50 (1788639350) [ 2906.180450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2906.192686] Lustre: *** cfs_fail_loc=197, val=0*** [ 2921.760843] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2926.883582] LustreError: 103448:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 4: rc = -110 [ 2932.123972] Lustre: *** cfs_fail_loc=197, val=0*** [ 2932.130173] Lustre: Skipped 65 previous similar messages [ 2952.672397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2952.683413] Lustre: Skipped 4 previous similar messages [ 2960.529244] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2970.069347] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2972.761572] Lustre: server umount lustre-OST0000 complete [ 2982.637883] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:17:15 (1788639435) [ 3004.730166] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3005.233109] LustreError: 105070:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3005.260827] LustreError: 105070:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 3010.239340] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3020.660555] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3024.842169] LustreError: 105101:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3024.865456] LustreError: 105101:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3024.883938] LustreError: 105101:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3025.399166] Lustre: server umount lustre-MDT0000 complete [ 3058.524091] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 3069.963881] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3110.891039] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3111.220366] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3111.245179] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3111.355080] Lustre: lustre-MDT0000: new disk, initializing [ 3111.470993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3115.644778] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3126.380975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3126.518501] Lustre: 109275:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3126.799949] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3131.568249] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3140.610341] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3146.946151] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3154.491813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3154.916472] Lustre: 111245:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3161.759145] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3164.904753] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3172.242791] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3177.378161] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3177.564133] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3183.079063] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3183.343571] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3185.636267] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3185.639653] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3185.641861] LustreError: Skipped 2 previous similar messages [ 3185.646117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3185.646125] Lustre: Skipped 13 previous similar messages [ 3185.696171] Lustre: Skipped 5 previous similar messages [ 3191.795914] Lustre: server umount lustre-OST0000 complete [ 3201.986994] Lustre: server umount lustre-MDT0000 complete [ 3206.141113] LustreError: 111255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639661 with bad export cookie 5708948535881992171 [ 3206.155694] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3206.160856] LustreError: 111255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3206.778682] Lustre: server umount lustre-MDT0001 complete [ 3217.765195] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:21:10 (1788639670) [ 3228.838769] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3232.054352] Key type lgssc unregistered [ 3232.444649] LNet: 113733:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3232.456364] LNetError: 113733:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3232.468645] LNet: Removed LNI 192.168.203.116@tcp [ 3233.436519] Key type .llcrypt unregistered [ 3233.438775] Key type ._llcrypt unregistered [ 3253.420786] Key type ._llcrypt registered [ 3253.423996] Key type .llcrypt registered [ 3253.564739] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 3271.487655] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3272.632170] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3272.794321] alg: No test for adler32 (adler32-zlib) [ 3273.848609] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 3274.081380] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3275.776346] Key type lgssc registered [ 3277.120717] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3319.683595] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3319.723812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3320.982291] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3321.008025] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3321.087658] Lustre: lustre-MDT0000: new disk, initializing [ 3321.177663] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3321.192737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3326.621112] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3339.414782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3339.714737] Lustre: lustre-OST0000: new disk, initializing [ 3339.719166] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3339.724511] Lustre: Skipped 1 previous similar message [ 3339.732805] Lustre: 117908:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3339.834429] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3345.555573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3345.572365] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3345.611899] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3347.780570] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3359.323951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3359.516404] Lustre: lustre-OST0001: new disk, initializing [ 3359.525882] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3359.532046] Lustre: 118771:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3359.651631] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3366.019420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3366.026678] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3366.063494] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3366.217318] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3376.098359] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3376.118184] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3381.371821] Lustre: server umount lustre-MDT0000 complete [ 3388.854992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3393.469918] LustreError: 117907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639849 with bad export cookie 7968888945224829593 [ 3393.484978] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3393.488193] LustreError: 117907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3393.598701] Lustre: server umount lustre-OST0000 complete [ 3397.553963] Lustre: server umount lustre-OST0001 complete [ 3407.026388] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 16:24:20 (1788639860) [ 3434.158964] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 16:24:46 (1788639886) [ 3446.953149] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3477.062809] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 16:25:29 (1788639929) [ 3517.698552] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 3527.734797] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3567.902893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3568.176517] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3568.235123] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3568.394305] Lustre: lustre-MDT0000: new disk, initializing [ 3568.530927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3568.559356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3574.788835] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3586.948541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3587.156071] Lustre: 126525:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3587.274897] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3587.286623] Lustre: Skipped 1 previous similar message [ 3587.477130] Lustre: lustre-MDT0001: new disk, initializing [ 3587.650950] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3587.742612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3587.758740] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3593.173061] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3602.822893] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3608.926528] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3616.264988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3616.588687] Lustre: lustre-OST0000: new disk, initializing [ 3616.594590] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3616.605340] Lustre: 128494:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3616.718344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3622.472123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3622.482840] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3622.626601] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3622.769117] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3632.538859] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3636.980900] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3637.166267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3641.446298] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3641.709641] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3648.481387] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3648.485319] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3648.501734] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3648.508362] Lustre: Skipped 2 previous similar messages [ 3648.526310] Lustre: Skipped 2 previous similar messages [ 3649.748905] Lustre: server umount lustre-OST0000 complete [ 3654.114127] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3654.132308] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3654.150602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3658.725843] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3658.734824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3658.745781] Lustre: Skipped 1 previous similar message [ 3660.221461] Lustre: server umount lustre-MDT0000 complete [ 3663.845698] LustreError: 126537:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3663.869661] LustreError: 126537:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3664.163143] LustreError: 126517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640119 with bad export cookie 7968888945224830818 [ 3664.170792] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3664.177951] LustreError: 126517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3664.551663] Lustre: server umount lustre-MDT0001 complete [ 3673.479558] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 16:28:46 (1788640126) [ 3679.937393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3680.411909] LustreError: 130907:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3680.486352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3684.583423] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3685.858166] LustreError: 130908:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3690.984232] LustreError: 130907:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3693.590380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3698.917768] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3708.935426] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3715.477454] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3722.317366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3722.706205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3722.718719] Lustre: Skipped 1 previous similar message [ 3729.181485] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3737.412827] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3745.429765] Lustre: *** cfs_fail_loc=159, val=0*** [ 3760.780044] Lustre: lustre-MDT0000: Client a21d31ec-5298-442c-a2bd-cc6d4eb57577 (at 192.168.203.16@tcp) reconnecting [ 3762.319106] Lustre: *** cfs_fail_loc=159, val=0*** [ 3762.323798] Lustre: Skipped 5 previous similar messages [ 3777.696469] Lustre: lustre-MDT0000: Client a21d31ec-5298-442c-a2bd-cc6d4eb57577 (at 192.168.203.16@tcp) reconnecting [ 3784.798877] Lustre: server umount lustre-OST0000 complete [ 3791.330538] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3791.331939] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3791.338683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3791.363887] Lustre: Skipped 1 previous similar message [ 3798.606975] Lustre: server umount lustre-MDT0000 complete [ 3801.572689] LustreError: 133399:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3801.572773] Lustre: 115109:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788640241/real 1788640241] req@00000000f75c04d1 x1875524692585344/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1788640257 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3801.599743] LustreError: 133399:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3801.672202] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3801.687618] Lustre: Skipped 1 previous similar message [ 3803.093329] LustreError: 130889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640258 with bad export cookie 7968888945224831700 [ 3803.105460] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3803.111105] LustreError: 130889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3803.675532] Lustre: server umount lustre-MDT0001 complete [ 3815.071803] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3818.664504] Key type lgssc unregistered [ 3819.104743] LNet: 134691:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3819.109756] LNetError: 134691:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3819.126680] LNet: Removed LNI 192.168.203.116@tcp [ 3820.317518] Key type .llcrypt unregistered [ 3820.323301] Key type ._llcrypt unregistered [ 3845.170396] Key type ._llcrypt registered [ 3845.174979] Key type .llcrypt registered [ 3845.264452] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3846.240208] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3846.404202] alg: No test for adler32 (adler32-zlib) [ 3847.495952] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 3847.853743] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3849.550337] Key type lgssc registered [ 3850.532950] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3857.854793] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3857.874355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3859.263595] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3863.436885] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3868.651687] LustreError: 136102:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3868.676274] LustreError: 136102:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3871.696754] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3872.248304] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3877.170436] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3884.696749] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3891.486918] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3899.044948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3899.435476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3903.547764] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3907.037269] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3917.044661] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3927.553301] Lustre: *** cfs_fail_loc=159, val=0*** [ 3943.169536] Lustre: lustre-MDT0000: Client d8a44b4e-18a2-4bf0-a215-0ef20f5ef79f (at 192.168.203.16@tcp) reconnecting [ 3945.750839] Lustre: *** cfs_fail_loc=159, val=0*** [ 3962.082716] Lustre: lustre-MDT0001: Client d8a44b4e-18a2-4bf0-a215-0ef20f5ef79f (at 192.168.203.16@tcp) reconnecting [ 3964.362171] Lustre: *** cfs_fail_loc=159, val=0*** [ 3964.363787] Lustre: Skipped 3 previous similar messages [ 3979.977787] Lustre: lustre-MDT0001: Client d8a44b4e-18a2-4bf0-a215-0ef20f5ef79f (at 192.168.203.16@tcp) reconnecting [ 3984.686453] Lustre: 139145:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3988.038122] Lustre: *** cfs_fail_loc=159, val=0*** [ 3988.052822] Lustre: Skipped 4 previous similar messages [ 4004.047798] Lustre: lustre-MDT0000: Client b126d2ee-71b0-4a9a-928d-907665c585fc (at 192.168.203.16@tcp) reconnecting [ 4006.253796] Lustre: *** cfs_fail_loc=159, val=0*** [ 4006.269700] Lustre: Skipped 14 previous similar messages [ 4021.969262] Lustre: lustre-MDT0000: Client b126d2ee-71b0-4a9a-928d-907665c585fc (at 192.168.203.16@tcp) reconnecting [ 4023.448946] Lustre: 139436:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4039.650178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4039.653695] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4039.682864] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4039.687186] Lustre: Skipped 1 previous similar message [ 4044.770175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4044.778839] Lustre: Skipped 1 previous similar message [ 4045.607492] Lustre: server umount lustre-OST0000 complete [ 4049.890855] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4049.894977] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4049.907502] Lustre: Skipped 2 previous similar messages [ 4055.010128] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4055.014789] Lustre: Skipped 2 previous similar messages [ 4055.540954] Lustre: server umount lustre-MDT0000 complete [ 4059.367112] LustreError: 139266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640515 with bad export cookie 12006766730942901521 [ 4059.370883] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4059.384588] LustreError: 139266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4059.954425] Lustre: server umount lustre-MDT0001 complete [ 4070.648476] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4073.317753] Key type lgssc unregistered [ 4073.604600] LNet: 140516:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4073.612399] LNetError: 140516:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4073.628118] LNet: Removed LNI 192.168.203.116@tcp [ 4074.393504] Key type .llcrypt unregistered [ 4074.398975] Key type ._llcrypt unregistered [ 4102.790895] Key type ._llcrypt registered [ 4102.796689] Key type .llcrypt registered [ 4103.011404] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4105.444172] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4105.463174] alg: No test for adler32 (adler32-zlib) [ 4106.634302] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 4107.054921] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4108.865885] Key type lgssc registered [ 4110.434387] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4118.925903] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4118.967620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4120.545810] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4125.875269] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4130.786265] LustreError: 141928:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4130.811244] LustreError: 141928:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4131.816885] LustreError: 141929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4136.536785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4137.051547] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4141.765461] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4149.600086] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4155.057519] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4162.471409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4162.927230] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4167.036384] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4169.438718] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4178.044976] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4181.365494] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4188.548795] Lustre: 144508:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4191.837956] Lustre: 144558:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4193.268833] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4194.096316] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4198.010795] Lustre: 144704:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4198.766181] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4199.727056] Lustre: 144803:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4207.075490] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4207.079075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4207.097674] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4207.103940] Lustre: Skipped 1 previous similar message [ 4210.115218] Lustre: server umount lustre-OST0000 complete [ 4217.315526] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4217.322566] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4217.337774] Lustre: Skipped 1 previous similar message [ 4217.353043] Lustre: Skipped 2 previous similar messages [ 4220.131815] Lustre: server umount lustre-MDT0000 complete [ 4222.436724] LustreError: 141929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4222.461098] LustreError: 141929:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4223.895888] LustreError: 141910:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640679 with bad export cookie 15487186000896922721 [ 4223.902112] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4223.906889] LustreError: 141910:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4224.245028] Lustre: server umount lustre-MDT0001 complete [ 4235.378259] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4239.261643] Key type lgssc unregistered [ 4239.566940] LNet: 145862:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4239.573454] LNetError: 145862:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4239.593987] LNet: Removed LNI 192.168.203.116@tcp [ 4240.598495] Key type .llcrypt unregistered [ 4240.601109] Key type ._llcrypt unregistered [ 4270.593113] Key type ._llcrypt registered [ 4270.594864] Key type .llcrypt registered [ 4270.695710] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4271.701210] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4271.734611] alg: No test for adler32 (adler32-zlib) [ 4272.842348] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 4273.119294] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4274.824231] Key type lgssc registered [ 4276.065334] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4284.236527] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4284.280129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4285.821586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4290.699616] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4296.162801] LustreError: 147279:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4296.184046] LustreError: 147279:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4300.520254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4300.951366] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4305.734221] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4314.114922] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4319.843726] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4327.388623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4327.863994] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4330.931169] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4334.223162] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4341.663221] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4346.309479] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4351.797203] Lustre: *** cfs_fail_loc=159, val=0*** [ 4368.082113] Lustre: lustre-MDT0000: Client a8e7d458-8c51-4d7a-bc50-899f64e155db (at 192.168.203.16@tcp) reconnecting [ 4377.058889] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4377.063742] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4377.070027] LustreError: Skipped 1 previous similar message [ 4377.072148] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4377.120035] Lustre: Skipped 1 previous similar message [ 4380.407724] Lustre: server umount lustre-OST0000 complete [ 4387.299516] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4387.316778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4387.317561] Lustre: Skipped 1 previous similar message [ 4387.333390] Lustre: Skipped 2 previous similar messages [ 4391.896619] Lustre: server umount lustre-MDT0000 complete [ 4392.419844] LustreError: 147278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4392.443746] LustreError: 147278:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4396.143130] LustreError: 147257:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640851 with bad export cookie 13779067823587734814 [ 4396.150808] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4396.154245] LustreError: 147257:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4396.591599] Lustre: server umount lustre-MDT0001 complete [ 4408.505597] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4411.788489] Key type lgssc unregistered [ 4412.147872] LNet: 150948:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4412.161752] LNetError: 150948:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4412.180720] LNet: Removed LNI 192.168.203.116@tcp [ 4412.974154] Key type .llcrypt unregistered [ 4412.979768] Key type ._llcrypt unregistered [ 4442.221467] Key type ._llcrypt registered [ 4442.229414] Key type .llcrypt registered [ 4442.466141] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4443.892416] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4444.135098] alg: No test for adler32 (adler32-zlib) [ 4445.359868] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 4445.704387] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4447.464278] Key type lgssc registered [ 4448.766186] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4456.869663] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4456.887065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4458.366420] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4462.513392] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4468.709364] LustreError: 152360:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4468.730020] LustreError: 152360:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4471.003382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4471.402413] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4475.553582] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4482.148067] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4487.973484] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4494.666032] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4495.233680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4497.274182] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4501.893476] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4503.414930] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4509.464690] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4530.745815] Lustre: lustre-OST0000: evicting 19507f60-27df-42d6-88e0-4a0203ff4f84 (at 192.168.203.16@tcp) by administrative request [ 4530.755645] Lustre: Skipped 1 previous similar message [ 4559.842121] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4559.861578] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4559.876770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4562.919769] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4562.920822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4565.292395] Lustre: server umount lustre-OST0000 complete [ 4573.155349] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4573.163900] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4573.173560] Lustre: Skipped 1 previous similar message [ 4573.189139] Lustre: Skipped 2 previous similar messages [ 4575.614745] Lustre: server umount lustre-MDT0000 complete [ 4578.284874] LustreError: 154577:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4578.322146] LustreError: 154577:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4579.892205] LustreError: 152339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788641035 with bad export cookie 3600272006771272798 [ 4579.897509] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4579.903146] LustreError: 152339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4580.245921] Lustre: server umount lustre-MDT0001 complete [ 4591.423533] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4594.086389] Key type lgssc unregistered [ 4594.388971] LNet: 156425:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4594.407624] LNetError: 156425:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4594.422514] LNet: Removed LNI 192.168.203.116@tcp [ 4595.270692] Key type .llcrypt unregistered [ 4595.272377] Key type ._llcrypt unregistered [ 4668.283793] Key type ._llcrypt registered [ 4668.288655] Key type .llcrypt registered [ 4668.385339] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4669.227574] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4669.283172] alg: No test for adler32 (adler32-zlib) [ 4670.341382] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 4670.519889] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4672.192377] Key type lgssc registered [ 4673.408213] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4681.335994] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4681.393336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4682.842337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4687.136201] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4692.976796] LustreError: 158913:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4692.996182] LustreError: 158913:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4695.602954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4696.134439] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4700.890409] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4709.924829] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4716.653925] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4724.650861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4724.974949] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4728.057087] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4731.714598] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4738.926108] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4741.367391] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4753.890099] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4753.913765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4753.930682] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4756.963745] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4756.966088] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4758.798452] Lustre: server umount lustre-OST0000 complete [ 4767.210841] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4767.215326] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4767.229077] Lustre: Skipped 1 previous similar message [ 4767.246018] Lustre: Skipped 2 previous similar messages [ 4770.112955] Lustre: server umount lustre-MDT0000 complete [ 4772.325453] LustreError: 158913:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4772.346612] LustreError: 158913:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4774.048131] LustreError: 158893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788641229 with bad export cookie 14288243867940453105 [ 4774.052557] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4774.059235] LustreError: 158893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4774.323699] Lustre: server umount lustre-MDT0001 complete [ 4785.775154] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4789.501532] Key type lgssc unregistered [ 4789.861192] LNet: 162474:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4789.874101] LNetError: 162474:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4790.891700] LNet: Removed LNI 192.168.203.116@tcp [ 4792.022164] Key type .llcrypt unregistered [ 4792.025070] Key type ._llcrypt unregistered [ 4816.738698] Key type ._llcrypt registered [ 4816.749347] Key type .llcrypt registered [ 4816.866975] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4818.165144] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4818.213738] alg: No test for adler32 (adler32-zlib) [ 4819.609713] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 4820.049339] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4821.792177] Key type lgssc registered [ 4823.598658] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4839.536984] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 16:48:12 (1788641292) [ 4855.398616] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 4866.247535] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4902.814296] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4902.846411] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4904.065562] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4904.105537] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4904.208113] Lustre: lustre-MDT0000: new disk, initializing [ 4904.289216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4904.310402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4908.774906] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4919.730491] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4919.907145] Lustre: 166731:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4919.982660] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4919.996251] Lustre: Skipped 1 previous similar message [ 4920.104371] Lustre: lustre-MDT0001: new disk, initializing [ 4920.184925] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4920.226128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4920.241511] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4925.414891] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4934.913811] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4940.488993] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4947.250604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4947.527974] Lustre: lustre-OST0000: new disk, initializing [ 4947.533867] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4947.542992] Lustre: 168697:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4947.652902] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4954.658043] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4955.212331] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4955.231599] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4955.328392] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4964.187468] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4976.104696] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4976.108186] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4976.124858] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4976.129325] LustreError: Skipped 1 previous similar message [ 4976.145609] Lustre: Skipped 1 previous similar message [ 4980.688858] Lustre: server umount lustre-OST0000 complete [ 4986.340962] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4986.342251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4986.353376] Lustre: Skipped 2 previous similar messages [ 4990.779727] Lustre: server umount lustre-MDT0000 complete [ 4994.186283] LustreError: 166723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788641449 with bad export cookie 363864840121853072 [ 4994.191390] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4994.197415] LustreError: 166723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4994.474544] Lustre: server umount lustre-MDT0001 complete [ 5005.909205] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 5008.588764] Key type lgssc unregistered [ 5008.847641] LNet: 170604:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5008.858365] LNetError: 170604:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5009.896406] LNet: Removed LNI 192.168.203.116@tcp [ 5010.792360] Key type .llcrypt unregistered [ 5010.799489] Key type ._llcrypt unregistered [ 5037.513824] Key type ._llcrypt registered [ 5037.521568] Key type .llcrypt registered [ 5037.679061] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 5039.213584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5039.355423] alg: No test for adler32 (adler32-zlib) [ 5040.420424] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 5040.683049] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 5042.496980] Key type lgssc registered [ 5043.968730] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5052.080273] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5052.095621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5053.634865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5058.526120] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 5064.165493] LustreError: 172016:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5064.182996] LustreError: 172016:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5067.305138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5067.667926] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5071.693659] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 5078.942710] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5084.349814] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5090.211432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5090.553180] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5096.423228] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 5098.736881] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5103.936928] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5109.778332] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5109.792611] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5109.805434] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5109.810887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.821776] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5110.603768] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5110.629146] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5110.644499] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5110.653955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.676156] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5111.613240] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5111.630996] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5111.650579] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5111.655481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.451485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.478340] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5112.486612] Lustre: Skipped 1 previous similar message [ 5113.330299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.126943] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5114.133848] Lustre: Skipped 2 previous similar messages [ 5114.156874] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5114.168285] Lustre: Skipped 2 previous similar messages [ 5114.171905] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5114.187358] Lustre: Skipped 2 previous similar messages [ 5114.191200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.034839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.042946] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5115.050728] Lustre: Skipped 2 previous similar messages [ 5115.764518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.600252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.364794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.330837] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5118.349289] Lustre: Skipped 4 previous similar messages [ 5118.382320] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5118.404320] Lustre: Skipped 4 previous similar messages [ 5118.411575] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5118.419902] Lustre: Skipped 4 previous similar messages [ 5118.425708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.326071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.336387] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5119.345868] Lustre: Skipped 4 previous similar messages [ 5120.299884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.182374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.108216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.993644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.964330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.805810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.685497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.503246] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5126.508090] Lustre: Skipped 8 previous similar messages [ 5126.531816] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5126.541465] Lustre: Skipped 8 previous similar messages [ 5126.544976] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5126.553351] Lustre: Skipped 8 previous similar messages [ 5126.561632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.466590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.479753] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5127.488311] Lustre: Skipped 8 previous similar messages [ 5128.382132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.334881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.225603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.423745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.552385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.591304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.522086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.536719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.558192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.579271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.489693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.517875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.527177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.521472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.639987] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5142.648104] Lustre: Skipped 15 previous similar messages [ 5142.663523] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5142.679227] Lustre: Skipped 15 previous similar messages [ 5142.687099] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5142.696149] Lustre: Skipped 15 previous similar messages [ 5142.707079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.613837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.640651] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5143.648425] Lustre: Skipped 15 previous similar messages [ 5144.503486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.471986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.455197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.520523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.582871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.607511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.501782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.487692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.493474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.476828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.441188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.619368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.686570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.829184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.969690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.086755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.364545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.451650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.479572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.344635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.219993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.356296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.487689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.648267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.864600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.748108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.860856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.684992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.834674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.748914] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5174.762663] Lustre: Skipped 30 previous similar messages [ 5174.779170] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5174.792536] Lustre: Skipped 30 previous similar messages [ 5174.799529] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5174.804634] Lustre: Skipped 30 previous similar messages [ 5174.807637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.927552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.940255] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5175.951652] Lustre: Skipped 30 previous similar messages [ 5176.925450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.983736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.956341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.931416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.739605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.660383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.549737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.652638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.619605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.475161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.437072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.600153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.623823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.634301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.619087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.767674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.678582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.748343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.571110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.525831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.269110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.188890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.226851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.123628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.909808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.728097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.515754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.473120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.439522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.486669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.294688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.088679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.913917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.712189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.509046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.453209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.425117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.301565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.173700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.942098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.777422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.622495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.739123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.721454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.704287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.530460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.454726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.309272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.220584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.012550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.780966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.544964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.427792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.223168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.246851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.156080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.189767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.115092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.031069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.913308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.813866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.623730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.453031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.326775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.139413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.015223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.781195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.520023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.381256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.252064] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5239.259324] Lustre: Skipped 70 previous similar messages [ 5239.277567] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5239.286068] Lustre: Skipped 70 previous similar messages [ 5239.288929] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5239.294086] Lustre: Skipped 70 previous similar messages [ 5239.300303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.057133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.097454] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5240.103409] Lustre: Skipped 70 previous similar messages [ 5240.868482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.883109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.821515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.699586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.629810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.387139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.297192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.095895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.985892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.713703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.547832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.378578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.182731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.870535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.679640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.436375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.346750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.204404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.035719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.857452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.753678] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5257.766863] LustreError: 172036:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5257.769205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.869378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.985301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.823625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.671263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.688442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.527356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.363084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.239551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.081761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.919808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.851191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.538460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.322077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.044932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.806968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.524984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.309690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.088772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.991710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.888688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.788274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.520643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.332636] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5277.341767] LustreError: 172036:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5277.352746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.219035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.190087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.153965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.164609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.060560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.050300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.786081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.661251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.548053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.368729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.195019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.043276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.733355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.949398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.791885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.661495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.544062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.596466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.598221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.739883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.686077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.679772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.567894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.354686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.280030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.425120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.287833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.342116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.181860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.048959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.096105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.054365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.106943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.078152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.165646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.089057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.277073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.160356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.173717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.220664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.132433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.173949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.244623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.071687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.508475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.680267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.744637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.858405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.955338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.905112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.906451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.689532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.745555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.844765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.750069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.684415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.790524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.713513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.737754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.768271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.695319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.703174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.529697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.352431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.274469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.024450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.879768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.803177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.730212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.462700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.295449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.995919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.907732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.768520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.466668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.372072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.184841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.003986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.814945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.552983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.252635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.951579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.660997] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5354.666253] LustreError: 172036:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5354.677826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.441978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.144890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.853708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.618595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.424621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.214940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.008436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.881506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.764737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.695291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.665767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.583543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.633840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.577713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.334735] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5367.342171] Lustre: Skipped 142 previous similar messages [ 5367.356480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5367.368265] Lustre: Skipped 142 previous similar messages [ 5367.371624] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5367.375917] Lustre: Skipped 142 previous similar messages [ 5367.384105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.355454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.374846] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5368.384373] Lustre: Skipped 142 previous similar messages [ 5369.373237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.231566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.939926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.629356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.463850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.233252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.055954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.908029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.784588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.853605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.701558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.734822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.576628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.430758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.165655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.160890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.001881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.653722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.603601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.767688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.741703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.677986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.895343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.014579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.800234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.610533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.514996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.386281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.254592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.069732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.855850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.769677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.504708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.513052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.286301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.124698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.053898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.048277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.973823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.926617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.704298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.507965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.314764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.215677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.117361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.038227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.898419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.785819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.650072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.521661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.505247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.331824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.267822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.156047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.915762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.870696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.634994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.572421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.589471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.440388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.371609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.193724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.076958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.925182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.627617] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5425.636776] LustreError: 172036:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5425.645073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.302943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.048763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.997523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.820398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.571230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.386108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.116514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.845781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.639354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.392139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.195518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.044148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.750848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.638690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.550422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.625854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.550837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.655172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.564766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.659344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.468811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.327318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.121739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.991119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.960560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.920838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.699523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.546227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.356607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.218550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.202563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.285058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.215354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.203662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.126823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.024110] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5457.030293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.030560] LustreError: 172036:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5457.916176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.704781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.505094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.286837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.241946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.080862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.933762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.862796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.883764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.651198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.464659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.269815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.135792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.871570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.848635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.664726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.441404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.252329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.020793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.753460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.606348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.444075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.290843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.104872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.899646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.748180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.635719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.536893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.474782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.564476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.382100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.284710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.134867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.104620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.956587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.976758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.967660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.853814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.850315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.885694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.781944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.902804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.772846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.790326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.724365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.472201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.313042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.253331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.993482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.825071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.575375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.388098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.131187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.946549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.702438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.697668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.637678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.561122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.276890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.000686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.850529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.783141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.724437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.561765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.374274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.370781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.280728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.196778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.067304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.981682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.930640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.971641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.936744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.858919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.725085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.678884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.555166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.495386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.631298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.683487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.593407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.601670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.412825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.426351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.473893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.503617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.592858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.606676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.661939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.628561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.684511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.652744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.551710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.438369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.511959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.408304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.267774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.128574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.933463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.846957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.824701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.773782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.649380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.506620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.460707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.408843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.390691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.273858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.185762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.998900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.056899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.980287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.950916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.845360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.745828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.708890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.621217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.487573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.646697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.713320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.596249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.546336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.544273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.541631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.614190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.614941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.598718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.591265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.642210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.533748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.706464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.667399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.738295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.870366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.931544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.846452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.889400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.136525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.332580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.273828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.187041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.173286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.069310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.956449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.015221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.122454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.222204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.132556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.175307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.457518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.374517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.451456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.413310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.286080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.220718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.143463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.062566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.939795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.911815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.860169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.831658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.862906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.783114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.600511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.383340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.258913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.166583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.220604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.139053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.129945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.110883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.093341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.903778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.724261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.537713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.298546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.057449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.090869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.245837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.287733] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5624.292491] Lustre: Skipped 281 previous similar messages [ 5624.314295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5624.329129] Lustre: Skipped 281 previous similar messages [ 5624.334737] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5624.339350] Lustre: Skipped 281 previous similar messages [ 5624.345478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.256248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.273185] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5625.283264] Lustre: Skipped 281 previous similar messages [ 5626.164176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.118134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.086274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.943186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.840287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.607291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.433533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.418167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.395659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.260376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.075753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.048041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.876908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.709302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.517618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.261126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.117251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.782754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.637958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.456782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.366123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.353117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.497744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.488045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.445489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.169196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.992040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.674435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.362057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.142749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.896697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.576427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.321264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.995170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.716172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.484784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.288352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.071754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.901521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.755228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.609995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.531349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.492246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.416042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.206137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.932258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.881397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.799127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.775995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.653619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.728303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.747251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.794561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.836505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.860410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.835953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.969962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.988744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.982561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.919836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.086930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.302951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.313607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.197264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.169872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.259743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.200992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.246628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.046270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.269445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.645327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.945039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.951154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.912966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.024206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.147816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.960894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.767196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.637781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.715930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.696308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.612084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.953875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.190917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.128671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.238336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.351657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.301325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.233598] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5708.241884] LustreError: 172036:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5708.247696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.198285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.322068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.304901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.366537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.336726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.288753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.355960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.346838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.188369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.076243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.939065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.844132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.621980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.613818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.678166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.582589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.715727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.775227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.645476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.543914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.362095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.148742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.150212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.946647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.795875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.774482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.830994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.826623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.767528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.645838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.574145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.451524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.397651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.216852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.264502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.221102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.380073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.567896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.635261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.713530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.680539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.711703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.646934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.653812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.543215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.556288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.409472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.301360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.243282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.223640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.055372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.990643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.968907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.940248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.714711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.640968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.522282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.364229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.208960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.086593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.969887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.861712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.787097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.713857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.493648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.370940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.204640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.200907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.090304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.111629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.960895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.833206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.743939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.709091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.503052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.522847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.435277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.433460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.315538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.267610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.972214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.899273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.803241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.667510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.571828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.536553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.535062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.361205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.297652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.151850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.281132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.596253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.662285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.799383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.114956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.088445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.238389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.410286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.344532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.286335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.365717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.298938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.270831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.256417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.235757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.200462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.065124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.186051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.207122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.085916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.145891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.244110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.297583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.157476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.160192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.060146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.081138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.974456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.042771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.194611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.215995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.180239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.191726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.429080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.254475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.094833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.905440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.785100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.710214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.581676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.402064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.291361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.172554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.211122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.107163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.287452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.276606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.173784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.965837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.016920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.932359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.827326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.792334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.756891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.880102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.945962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.945420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.919967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.126165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.190875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.339898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.420490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.303839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.346244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.339262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.273232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.285058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.401472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.387730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.409654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.281568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.383560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.301268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.310521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.388424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.318255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.340911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.294817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.391241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.227343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.150925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.002503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.034467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.961208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.788977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.712712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.701322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.726235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.755129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.772457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.802967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.748900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.631074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.422119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.343634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.095480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.240790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.163852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.183317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.339695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.276277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.245049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.222882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.119797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.061465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.897704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.725877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.619285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.448763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.206032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.170038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.033808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.081082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.297984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.180955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.142232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.005880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.996068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.928269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.924018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.938092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.954958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.742037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.610880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.542122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.489417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.360868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.375087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.324128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.361687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.339401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.387684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.239850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.293039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.197801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.095342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.019457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.283402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.302730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.351367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.214747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.341730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.434666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.425522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.310523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.378659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.474104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.489806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.541753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.562207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.626982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.496816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.492547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.582988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.506954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.378240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.475517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.649061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.554566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.572112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.529897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.598753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.627496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.795426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.698363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.769822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.858256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.725312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.725289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.741936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.822972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.798420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.700378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.655717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.722768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.840643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.733321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.838758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.744531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.923044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.082539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.398705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.356587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.453935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.424642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.688310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.725699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.713947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.783674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.839983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.715412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.553927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.392929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.296865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.433640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.325265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.253200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.306171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.238943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.233922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.266753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.206562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.078269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.139844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.199336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.185753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.098499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.010854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.883768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.876804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.792057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.644359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.470593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.572514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.691476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.643516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.714560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.880770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.764820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.708912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.623725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.691078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.543634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.668611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.560829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.493908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.447635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.284575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.018789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.903889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.837074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.770877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.727697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.800602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.663701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.706986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.731455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.687301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.593195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.456933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.298647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.163400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.952499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.770784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.664182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.729825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.635703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.528458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.449670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.384164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.227362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.109857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.901086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.898385] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6040.915529] LustreError: 172036:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6040.927990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.050797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.076388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.263890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.923222] LustreError: 174312:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6439.949153] LustreError: 172036:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7151.074517] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7151.082263] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7151.093622] Lustre: Skipped 437 previous similar messages [ 7151.097376] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7151.102198] Lustre: Skipped 1 previous similar message [ 7154.153789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7154.163958] Lustre: Skipped 1 previous similar message [ 7156.722587] Lustre: server umount lustre-OST0000 complete [ 7161.832226] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7161.838454] LustreError: Skipped 1 previous similar message [ 7161.842564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7164.387151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7164.392131] Lustre: Skipped 1 previous similar message [ 7166.885036] Lustre: server umount lustre-MDT0000 complete [ 7169.508844] LustreError: 226148:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7169.521988] LustreError: 226148:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7170.534551] LustreError: 171996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788643626 with bad export cookie 17125094429789031558 [ 7170.538261] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7170.544416] LustreError: 171996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7170.882412] Lustre: server umount lustre-MDT0001 complete [ 7181.226912] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 7183.815064] Key type lgssc unregistered [ 7184.104609] LNet: 227287:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7184.117957] LNetError: 227287:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7184.133471] LNet: Removed LNI 192.168.203.116@tcp [ 7184.934392] Key type .llcrypt unregistered [ 7184.936806] Key type ._llcrypt unregistered [ 7210.440732] Key type ._llcrypt registered [ 7210.443191] Key type .llcrypt registered [ 7210.597628] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7212.011734] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7212.047809] alg: No test for adler32 (adler32-zlib) [ 7213.181800] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 7213.501024] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 7215.272163] Key type lgssc registered [ 7216.565256] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7224.198170] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7224.232381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7225.738328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7230.065485] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7236.074757] LustreError: 228704:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7236.091920] LustreError: 228704:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7237.959493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7238.392312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7242.505615] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7249.835868] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7255.824835] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7262.459359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7262.820397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7269.245041] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7271.041846] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7277.428383] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7282.442726] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7284.008779] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7284.026994] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7284.051238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7284.089462] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7284.091040] LustreError: 228723:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7286.761383] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7286.772169] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7286.775677] Lustre: Skipped 1 previous similar message [ 7286.787592] Lustre: Skipped 1 previous similar message [ 7289.523896] Lustre: lustre-OST0000: Not available for connect from 192.168.203.16@tcp (stopping) [ 7291.875800] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7291.896016] Lustre: Skipped 1 previous similar message [ 7292.563371] Lustre: server umount lustre-OST0000 complete [ 7304.176207] LustreError: 228723:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7329.425254] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7330.208850] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7336.721570] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7341.484647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.503147] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.513616] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7344.897144] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7346.524751] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7352.460265] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8c2d10b22800.ost_server_uuid 50 [ 7354.046928] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8c2d10b22800.ost_server_uuid in IDLE state after 0 sec [ 7359.458062] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7359.467770] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7359.484641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7362.017540] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7362.032456] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7364.346922] Lustre: server umount lustre-OST0000 complete [ 7371.745474] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7371.752716] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7371.765185] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7371.767977] Lustre: Skipped 2 previous similar messages [ 7374.370053] Lustre: server umount lustre-MDT0000 complete [ 7377.386020] LustreError: 228704:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7377.403279] LustreError: 228704:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7378.381397] LustreError: 233164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788643834 with bad export cookie 2988061663908534632 [ 7378.399965] LustreError: 233164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7378.402414] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7379.241913] Lustre: server umount lustre-MDT0001 complete [ 7392.722405] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 7396.454409] Key type lgssc unregistered [ 7396.921190] LNet: 233915:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7396.946522] LNetError: 233915:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7396.992592] LNet: Removed LNI 192.168.203.116@tcp [ 7398.162513] Key type .llcrypt unregistered [ 7398.166297] Key type ._llcrypt unregistered [ 7435.981939] Key type ._llcrypt registered [ 7435.986672] Key type .llcrypt registered [ 7436.145996] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7437.756757] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7437.820406] alg: No test for adler32 (adler32-zlib) [ 7438.951512] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 7439.243950] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 7441.056740] Key type lgssc registered [ 7442.032955] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7453.000474] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7453.035116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7454.562651] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7459.068486] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7464.933607] LustreError: 235639:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7464.944093] LustreError: 235639:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7468.436398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7468.771558] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7473.070871] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7476.182908] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7482.955921] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7483.371459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7489.785251] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7499.058704] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7499.209830] Lustre: lustre-OST0001: new disk, initializing [ 7499.213484] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7499.218696] Lustre: 237947:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7499.305735] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7500.352436] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7505.024919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7505.036061] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7505.178042] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7506.537470] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7519.268359] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7528.135799] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7545.825666] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7545.834933] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7545.868304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7546.346080] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7546.363214] Lustre: Skipped 1 previous similar message [ 7546.369793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7546.372956] Lustre: Skipped 3 previous similar messages [ 7551.459127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7551.471331] Lustre: Skipped 3 previous similar messages [ 7556.579363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7556.587649] Lustre: Skipped 3 previous similar messages [ 7559.141059] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7561.349340] Lustre: server umount lustre-MDT0000 complete [ 7561.711069] LustreError: 236989:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7561.743851] LustreError: 236989:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7565.595900] LustreError: 235620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788644021 with bad export cookie 3787648879568436453 [ 7565.607791] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7565.611189] LustreError: 235620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7566.002080] Lustre: server umount lustre-MDT0001 complete [ 7580.692093] Lustre: server umount lustre-OST0000 complete [ 7583.201352] Lustre: 234900:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788644022/real 1788644022] req@ffff97a75e6af800 x1875529059893248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788644038 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7583.269164] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7583.288728] Lustre: Skipped 1 previous similar message [ 7584.966190] Lustre: server umount lustre-OST0001 complete [ 7610.884977] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7621.956212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7622.451949] LustreError: 241059:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7622.529409] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7627.728964] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7627.746327] LustreError: 241060:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7632.879746] LustreError: 241059:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7637.392738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7637.703770] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7642.268676] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7645.405604] Lustre: 242201:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7652.937849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7660.065351] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7662.647665] LustreError: 242553:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7668.738796] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7669.652102] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7669.961428] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7669.977082] Lustre: Skipped 1 previous similar message [ 7675.392910] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7677.435981] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7685.738542] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7689.926460] Lustre: 244073:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7704.033697] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7704.047519] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7704.073922] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7708.763457] Lustre: server umount mylustre-MDT0000 complete [ 7711.207037] LustreError: 241056:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7711.238126] LustreError: 241056:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7712.851824] LustreError: 241035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788644168 with bad export cookie 3787648879568439232 [ 7712.854798] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7712.873875] LustreError: 241035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7713.405602] Lustre: server umount mylustre-MDT0001 complete [ 7727.546619] Lustre: server umount mylustre-OST0000 complete [ 7742.549465] Lustre: server umount mylustre-OST0001 complete [ 7765.820721] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7776.053798] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7776.502821] LustreError: 246382:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7776.616912] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7781.645781] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7791.369713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7798.296217] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7802.249852] Lustre: 247524:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7810.781603] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7820.189683] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7821.781562] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7829.152500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7834.626368] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7836.148081] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7844.852742] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7848.684717] Lustre: 249397:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7858.566916] Lustre: server umount tfs-MDT0000 complete [ 7858.658750] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7858.676502] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7858.682513] Lustre: Skipped 3 previous similar messages [ 7858.686177] LustreError: 247932:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7858.692697] LustreError: 247932:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 7863.911159] LustreError: 246357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788644319 with bad export cookie 3787648879568441115 [ 7863.912886] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7863.931500] LustreError: 246357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7864.607617] Lustre: server umount tfs-MDT0001 complete [ 7881.208689] Lustre: server umount tfs-OST0000 complete [ 7882.080927] Lustre: 234903:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788644321/real 1788644321] req@ffff97a74fb71f80 x1875529060007424/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788644337 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7882.123112] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7882.144666] Lustre: Skipped 3 previous similar messages [ 7885.281110] Lustre: 234901:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788644324/real 1788644324] req@ffff97a74fb71500 x1875529060007680/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788644340 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7885.751258] Lustre: server umount tfs-OST0001 complete [ 7916.071671] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7928.199404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7929.054177] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7929.061234] Lustre: Skipped 3 previous similar messages [ 7934.084666] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7943.699492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7949.273416] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7952.643411] Lustre: 252793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7959.903868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7966.558980] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7973.778593] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7976.116360] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7981.572952] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7982.958615] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7991.017565] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7994.876681] Lustre: 254663:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8002.688618] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 17:40:55 (1788644455) [ 8020.975374] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8020.990873] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8021.012826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8021.023082] Lustre: Skipped 4 previous similar messages [ 8025.453650] Lustre: server umount lustre-MDT0000 complete [ 8027.620142] LustreError: 251651:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8027.637550] LustreError: 251651:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 15 previous similar messages [ 8028.705583] LustreError: 251626:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788644484 with bad export cookie 3787648879568442788 [ 8028.719174] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8028.724461] LustreError: 251626:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8035.265794] Lustre: server umount lustre-MDT0001 complete [ 8045.594957] Lustre: server umount lustre-OST0000 complete [ 8055.695152] Lustre: server umount lustre-OST0001 complete [ 8063.995747] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 8075.949206] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8120.379644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8120.671433] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8120.697152] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8120.808692] Lustre: lustre-MDT0000: new disk, initializing [ 8120.890723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8120.900463] Lustre: Skipped 3 previous similar messages [ 8120.920409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8126.177463] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8138.336151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8138.486114] Lustre: 257848:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8138.607658] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8138.614918] Lustre: Skipped 1 previous similar message [ 8138.752386] Lustre: lustre-MDT0001: new disk, initializing [ 8138.955924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8138.976782] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8145.028901] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8159.833406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8160.079513] Lustre: lustre-OST0000: new disk, initializing [ 8160.082252] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8160.086903] Lustre: 259433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8162.174140] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8162.185154] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8162.300825] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8167.156678] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8179.377475] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8179.553026] Lustre: lustre-OST0001: new disk, initializing [ 8179.556894] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8179.564081] Lustre: 260303:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8181.775787] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8181.787939] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8181.848101] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8187.000569] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8200.856751] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 17:44:13 (1788644653) [ 8216.975645] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 17:44:29 (1788644669) [ 8232.935610] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8232.951623] Lustre: Skipped 5 previous similar messages [ 8232.959521] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8232.965708] Lustre: Skipped 9 previous similar messages [ 8238.260196] Lustre: server umount lustre-OST0000 complete [ 8248.920156] Lustre: server umount lustre-MDT0000 complete [ 8252.063471] LustreError: 258646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788644707 with bad export cookie 3787648879568444335 [ 8252.075647] LustreError: 258646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8252.077643] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8258.535554] Lustre: server umount lustre-MDT0001 complete [ 8269.663823] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8344.880142] Lustre: server umount lustre-OST0001 complete [ 8351.646641] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 8368.698840] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8405.604568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8405.872330] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8405.957248] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8406.066689] Lustre: lustre-MDT0000: new disk, initializing [ 8406.135323] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8406.143647] Lustre: Skipped 3 previous similar messages [ 8406.165316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8410.818978] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8421.038191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8421.177765] Lustre: 265156:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8421.227424] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8421.230985] Lustre: Skipped 1 previous similar message [ 8421.300716] Lustre: lustre-MDT0001: new disk, initializing [ 8421.457018] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8421.468800] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8426.536763] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8436.322169] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8441.848714] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8449.221219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8449.612314] Lustre: lustre-OST0000: new disk, initializing [ 8449.617875] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8449.629931] Lustre: 267126:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8450.979527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8450.997889] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8451.178926] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8456.815852] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8466.966499] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8476.641748] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8476.653373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8476.657507] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8476.660218] Lustre: Skipped 6 previous similar messages [ 8476.682021] Lustre: Skipped 11 previous similar messages [ 8481.303791] Lustre: server umount lustre-OST0000 complete [ 8491.684180] Lustre: server umount lustre-MDT0000 complete [ 8492.003287] LustreError: 265167:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8492.027494] LustreError: 265167:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8495.525420] LustreError: 265147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788644951 with bad export cookie 3787648879568445812 [ 8495.530726] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8495.541199] LustreError: 265147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8495.941287] Lustre: server umount lustre-MDT0001 complete [ 8506.165720] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8509.071701] Key type lgssc unregistered [ 8509.366771] LNet: 268885:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8509.374604] LNetError: 268885:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8509.389828] LNet: Removed LNI 192.168.203.116@tcp [ 8510.170145] Key type .llcrypt unregistered [ 8510.173309] Key type ._llcrypt unregistered [ 8536.570632] Key type ._llcrypt registered [ 8536.576193] Key type .llcrypt registered [ 8536.730398] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8537.917192] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8538.027359] alg: No test for adler32 (adler32-zlib) [ 8539.111840] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 8539.438037] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 8541.144187] Key type lgssc registered [ 8542.269724] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8550.062796] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8550.080560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8551.652434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8557.049368] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8562.155127] LustreError: 270345:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8562.202958] LustreError: 270345:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8565.831340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8566.300033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8570.900988] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8580.508879] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8587.067372] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8595.981738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8596.472297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8597.522338] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8604.605482] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8612.558974] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8613.489846] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8615.910790] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8615.922445] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8615.930912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8615.951420] Lustre: Skipped 1 previous similar message [ 8617.404394] Lustre: server umount lustre-OST0000 complete [ 8621.025872] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8621.047580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8621.061584] Lustre: Skipped 2 previous similar messages [ 8626.147577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8626.153574] Lustre: Skipped 1 previous similar message [ 8631.266619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8631.283612] Lustre: Skipped 1 previous similar message [ 8635.361659] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8635.364925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8635.390975] Lustre: Skipped 1 previous similar message [ 8635.559879] Lustre: server umount lustre-MDT0000 complete [ 8638.627778] LustreError: 270325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788645094 with bad export cookie 17107959363919050152 [ 8638.637959] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8638.641209] LustreError: 270325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8638.984433] Lustre: server umount lustre-MDT0001 complete [ 8649.193854] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8652.142572] Key type lgssc unregistered [ 8652.539549] LNet: 273927:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8652.546161] LNetError: 273927:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8652.567425] LNet: Removed LNI 192.168.203.116@tcp [ 8653.391810] Key type .llcrypt unregistered [ 8653.394267] Key type ._llcrypt unregistered [ 8671.200871] Key type ._llcrypt registered [ 8671.208182] Key type .llcrypt registered [ 8671.371379] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8672.709313] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8672.736596] alg: No test for adler32 (adler32-zlib) [ 8673.860524] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 8674.213970] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 8676.000300] Key type lgssc registered [ 8677.052546] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8685.806917] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8685.820560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8701.280414] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8706.400354] LustreError: 275118:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 4: rc = -110 [ 8733.032459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8738.439938] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8745.346594] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8747.328962] Lustre: server umount lustre-OST0000 complete [ 8765.707827] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8769.676437] Key type lgssc unregistered [ 8770.023886] LNet: 276748:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8770.028666] LNetError: 276748:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8770.062767] LNet: Removed LNI 192.168.203.116@tcp [ 8771.222301] Key type .llcrypt unregistered [ 8771.225649] Key type ._llcrypt unregistered [ 8813.544327] Key type ._llcrypt registered [ 8813.546628] Key type .llcrypt registered [ 8813.668995] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8814.807597] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8814.899190] alg: No test for adler32 (adler32-zlib) [ 8815.883404] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 8816.319591] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 8818.198146] Key type lgssc registered [ 8820.050283] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8836.921098] loop0: detected capacity change from 0 to 204800000 [ 8837.202175] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8837.212980] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8837.791960] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8838.993023] loop0: detected capacity change from 0 to 204800000 [ 8839.324403] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8839.361195] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8839.692331] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8840.556962] loop0: detected capacity change from 0 to 204800000 [ 8840.891697] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8840.910524] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8841.119246] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8842.600942] loop0: detected capacity change from 0 to 204800000 [ 8842.819373] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8842.859396] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8843.026149] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8843.960609] loop0: detected capacity change from 0 to 204800000 [ 8843.984564] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8848.503418] loop0: detected capacity change from 0 to 204800000 [ 8848.540033] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8852.628798] loop0: detected capacity change from 0 to 204800000 [ 8852.652826] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8856.878025] loop0: detected capacity change from 0 to 204800000 [ 8856.917096] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8860.801277] loop0: detected capacity change from 0 to 204800000 [ 8860.895419] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8860.954339] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8861.020948] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8861.035539] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8861.047745] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8861.064899] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8861.073713] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8861.089588] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8861.099466] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8861.108741] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8861.116258] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8861.125602] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8861.137973] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8861.145955] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8861.160431] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8861.170363] Lustre: 280161:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8866.598739] Lustre: server umount MGS complete [ 8867.254921] loop0: detected capacity change from 0 to 204800000 [ 8867.327578] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8867.660078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8867.697230] Lustre: lustre-MDT0000: Aborting client recovery [ 8868.438487] loop1: detected capacity change from 0 to 204800000 [ 8868.551298] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8868.631320] Lustre: 280593:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8868.641647] Lustre: 280593:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8868.664258] Lustre: 280593:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8868.679306] Lustre: 280593:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8868.688945] Lustre: 280593:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8868.703513] Lustre: 280593:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8868.720848] Lustre: 280593:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8868.739216] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8868.930185] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8868.990903] Lustre: lustre-MDT0001: Aborting client recovery [ 8869.919647] loop2: detected capacity change from 0 to 204800000 [ 8870.024954] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8870.127685] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8870.146057] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8870.159056] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8870.173280] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8870.185539] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8870.197761] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8870.207480] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8870.344200] Lustre: Found index 0 for lustre-OST0000, updating log [ 8870.397214] Lustre: 280691:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8870.500566] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8870.520883] Lustre: lustre-OST0000: Aborting recovery [ 8871.376535] loop3: detected capacity change from 0 to 204800000 [ 8871.487069] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8871.597312] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8871.609873] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8871.637214] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8871.654702] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8871.669209] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8871.681784] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8871.689848] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8871.711389] Lustre: Found index 1 for lustre-OST0001, updating log [ 8871.742312] Lustre: 280763:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8871.811407] Lustre: lustre-OST0001: Aborting recovery [ 8877.052534] LustreError: 280536:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8878.058278] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8878.095271] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8878.114308] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8878.162635] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8879.081920] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8879.082959] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8880.279196] LustreError: 280503:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 255us: rc = -2 [ 8880.798984] LustreError: 280504:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 456us: rc = -2 [ 8880.821851] LustreError: 280504:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8894.434991] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8894.441147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8894.441926] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8894.448391] Lustre: Skipped 3 previous similar messages [ 8894.476617] Lustre: Skipped 3 previous similar messages [ 8899.554732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8899.569428] Lustre: Skipped 3 previous similar messages [ 8899.933948] Lustre: server umount lustre-MDT0000 complete [ 8900.775051] LustreError: 280487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788645356 with bad export cookie 12590364473277642172 [ 8900.781767] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8900.792720] LustreError: 280487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8904.673466] LustreError: 280994:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8904.673880] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8904.678490] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8904.700700] LustreError: 280994:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8904.770114] Lustre: Skipped 1 previous similar message [ 8905.697944] LustreError: 280994:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8905.727727] LustreError: 280994:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8907.198900] Lustre: server umount lustre-MDT0001 complete [ 8914.402369] Lustre: server umount lustre-OST0000 complete [ 8921.984971] Lustre: server umount lustre-OST0001 complete [ 8932.792106] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:56:25 (1788645385) [ 8964.940610] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 8975.843874] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9018.716160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9018.971936] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9018.993319] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9019.042680] Lustre: lustre-MDT0000: new disk, initializing [ 9019.108584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9019.114691] Lustre: Skipped 1 previous similar message [ 9019.131970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9024.321344] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9035.743380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9035.813878] Lustre: 284836:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9035.905156] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9035.911187] Lustre: Skipped 1 previous similar message [ 9035.976957] Lustre: lustre-MDT0001: new disk, initializing [ 9036.108155] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9036.191180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9036.199166] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9042.143862] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9056.189095] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9064.732567] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9073.829605] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9074.172621] Lustre: lustre-OST0000: new disk, initializing [ 9074.176353] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9074.196540] Lustre: 286805:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9074.336182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9080.897113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9080.904261] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9080.920263] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9081.026426] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9092.824144] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9098.591413] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9108.718289] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9115.518824] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9122.848472] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9163.241885] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9163.242790] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9163.249668] LustreError: Skipped 1 previous similar message [ 9163.252382] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9163.265687] Lustre: Skipped 1 previous similar message [ 9163.292727] Lustre: Skipped 4 previous similar messages [ 9167.468827] Lustre: server umount lustre-OST0000 complete [ 9173.474594] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9173.479361] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9173.488341] Lustre: Skipped 1 previous similar message [ 9177.541890] Lustre: server umount lustre-MDT0000 complete [ 9178.594505] LustreError: 284847:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9178.612059] LustreError: 284847:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9181.309851] LustreError: 284828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788645637 with bad export cookie 12590364473277644874 [ 9181.313415] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9181.323129] LustreError: 284828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9181.818314] Lustre: server umount lustre-MDT0001 complete [ 9190.383049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9194.827821] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9199.858918] Lustre: server umount MGS complete [ 9210.955206] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9211.641203] LustreError: 290638:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9211.720279] LustreError: 290651:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9211.838839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9216.887329] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9216.994652] LustreError: 290639:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9227.106743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9227.235912] LustreError: 290638:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9227.256140] LustreError: 290638:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9234.189571] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9243.226818] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9250.683987] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9260.023322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9260.617290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9260.625965] Lustre: Skipped 1 previous similar message [ 9269.030555] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9276.743940] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9312.740750] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9312.753751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9312.759417] Lustre: Skipped 1 previous similar message [ 9312.773940] Lustre: Skipped 2 previous similar messages [ 9318.588202] Lustre: server umount lustre-OST0000 complete [ 9322.982433] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9323.014910] Lustre: Skipped 1 previous similar message [ 9328.776037] Lustre: server umount lustre-MDT0000 complete [ 9333.020519] LustreError: 290618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788645788 with bad export cookie 12590364473277646813 [ 9333.026970] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9333.036964] LustreError: 290618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9333.425551] Lustre: server umount lustre-MDT0001 complete [ 9344.863854] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 9347.841971] Key type lgssc unregistered [ 9348.073297] LNet: 294584:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9348.090542] LNetError: 294584:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9348.111411] LNet: Removed LNI 192.168.203.116@tcp [ 9349.063940] Key type .llcrypt unregistered [ 9349.066122] Key type ._llcrypt unregistered [ 9391.155532] Key type ._llcrypt registered [ 9391.158702] Key type .llcrypt registered [ 9391.313551] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 9406.587169] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9408.416381] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9408.478342] alg: No test for adler32 (adler32-zlib) [ 9409.629242] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 9409.933883] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 9411.672179] Key type lgssc registered [ 9412.900885] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9457.007468] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9457.024953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9458.312682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9458.374694] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9458.537952] Lustre: lustre-MDT0000: new disk, initializing [ 9458.659640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9458.685889] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9463.686652] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9473.610351] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9473.706536] Lustre: 298540:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9473.762498] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9473.766374] Lustre: Skipped 1 previous similar message [ 9473.830245] Lustre: lustre-MDT0001: new disk, initializing [ 9473.936016] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9473.999845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9474.014138] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9477.972747] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9486.761297] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9492.599674] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9499.313471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9499.703875] Lustre: lustre-OST0000: new disk, initializing [ 9499.708847] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9499.715941] Lustre: 300508:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9499.830770] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9505.246310] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9510.015545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9510.030922] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9510.142712] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9515.368520] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9523.218907] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9536.569958] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9539.192632] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9546.937690] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9592.309815] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9592.314449] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9592.327402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9592.345842] Lustre: Skipped 1 previous similar message [ 9594.522264] Lustre: server umount lustre-OST0000 complete [ 9602.017457] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9602.035048] LustreError: Skipped 1 previous similar message [ 9602.042355] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9602.070715] Lustre: Skipped 1 previous similar message [ 9602.102063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9607.047907] Lustre: server umount lustre-MDT0000 complete [ 9611.487061] LustreError: 301365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646067 with bad export cookie 11898610898367672503 [ 9611.510752] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9611.515710] LustreError: 301365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9612.247018] Lustre: server umount lustre-MDT0001 complete [ 9620.808395] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9625.318755] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9630.015332] Lustre: server umount MGS complete [ 9641.435525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9641.895232] LustreError: 304279:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9641.915257] LustreError: 304279:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9641.933308] LustreError: 304292:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9642.000835] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9647.077373] LustreError: 304280:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9647.318569] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9652.194683] LustreError: 304279:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9657.319920] LustreError: 304280:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9657.447030] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9657.821837] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9663.380581] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9671.075056] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9676.065817] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9683.008215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9683.466313] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9689.971914] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9696.975116] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9722.351682] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9722.360115] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9722.370665] Lustre: Skipped 1 previous similar message [ 9722.379814] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9722.388358] Lustre: Skipped 1 previous similar message [ 9725.539517] Lustre: server umount lustre-OST0000 complete [ 9729.508924] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9729.515779] LustreError: Skipped 1 previous similar message [ 9729.520973] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9729.529976] Lustre: Skipped 1 previous similar message [ 9729.534830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9729.539641] Lustre: Skipped 2 previous similar messages [ 9733.602069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9733.603314] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9734.862147] Lustre: server umount lustre-MDT0000 complete [ 9738.299179] LustreError: 304259:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646194 with bad export cookie 11898610898367674323 [ 9738.316956] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9738.528587] Lustre: server umount lustre-MDT0001 complete [ 9747.399921] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 9749.253061] Key type lgssc unregistered [ 9749.442374] LNet: 308224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9749.451964] LNetError: 308224:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9749.472766] LNet: Removed LNI 192.168.203.116@tcp [ 9750.155888] Key type .llcrypt unregistered [ 9750.158507] Key type ._llcrypt unregistered [ 9771.421617] Key type ._llcrypt registered [ 9771.423697] Key type .llcrypt registered [ 9771.724583] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 9789.569595] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9791.100335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9791.180418] alg: No test for adler32 (adler32-zlib) [ 9792.889462] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [ 9793.126394] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 9794.776178] Key type lgssc registered [ 9796.541367] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9827.020266] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9842.342171] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9842.372879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9843.574611] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9843.608256] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9843.686510] Lustre: lustre-MDT0000: new disk, initializing [ 9843.758371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9843.778160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9848.392607] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9861.048263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9861.490848] Lustre: lustre-OST0000: new disk, initializing [ 9861.502417] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9861.516228] Lustre: Skipped 1 previous similar message [ 9861.527566] Lustre: 312273:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9861.661562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9867.445872] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9867.461857] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9867.498076] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9870.328175] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [10747.333870] Lustre: server umount lustre-MDT0000 complete [10750.859504] LustreError: 312272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647206 with bad export cookie 13509209338272525922 [10750.871452] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10761.346757] Lustre: server umount lustre-OST0000 complete [10787.471265] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [10798.858854] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [10844.512566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10844.738552] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10844.778314] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10844.872732] Lustre: lustre-MDT0000: new disk, initializing [10844.952542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10844.965496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10849.224759] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [10860.437711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10860.630551] Lustre: 316614:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10860.709583] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10860.716738] Lustre: Skipped 1 previous similar message [10860.862773] Lustre: lustre-MDT0001: new disk, initializing [10861.044197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10861.131723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10861.152029] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10865.934527] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [10876.032133] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10883.788565] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10891.388406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10891.602878] Lustre: lustre-OST0000: new disk, initializing [10891.606650] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10891.612356] Lustre: 318583:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10891.700062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10897.471395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10897.485516] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10897.589773] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10898.216788] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [10907.884394] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10912.244799] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10912.446585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10916.335285] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10916.536174] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10918.375676] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10918.376224] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10918.391798] LustreError: Skipped 1 previous similar message [10918.413410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10918.419286] Lustre: Skipped 1 previous similar message [10923.500116] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10923.518052] Lustre: Skipped 1 previous similar message [10924.361490] Lustre: server umount lustre-OST0000 complete [10927.586190] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10927.596592] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10927.616817] Lustre: Skipped 1 previous similar message [10927.622771] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10928.612303] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10933.733671] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10933.742396] Lustre: Skipped 3 previous similar messages [10933.837933] Lustre: server umount lustre-MDT0000 complete [10936.866437] LustreError: 316605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647392 with bad export cookie 13509209338275466741 [10936.875552] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10936.875867] LustreError: 316605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10937.138847] Lustre: server umount lustre-MDT0001 complete [10940.613035] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10942.242660] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 18:29:55 (1788647395) [10949.903185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10950.530558] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10955.754180] LustreError: 320899:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10955.793580] LustreError: 320899:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10956.060917] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [10960.868423] LustreError: 320898:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10965.464117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10965.757305] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10970.008170] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [10977.521511] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10983.444729] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10990.241872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10990.664290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10998.037647] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11006.555874] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11014.625094] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11014.846573] Lustre: lustre-OST0001: new disk, initializing [11014.857615] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11014.871824] Lustre: 323734:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11015.002112] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11021.541283] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11021.916528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11021.927586] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11021.996660] LustreError: 324112:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11023.021807] LustreError: 324112:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11023.043899] LustreError: 324112:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11024.107417] LustreError: 324117:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11026.221654] LustreError: 324117:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11026.242619] LustreError: 324117:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [11030.310295] LustreError: 324112:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11030.325755] LustreError: 324112:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [11031.268200] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11031.602865] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11037.365860] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8c2d07b2b000.ost_server_uuid 50 [11038.564493] LustreError: 324117:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11038.578757] LustreError: 324117:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [11038.930226] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8c2d07b2b000.ost_server_uuid in FULL state after 0 sec [11044.697949] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8c2d07b2b000.ost_server_uuid 50 [11046.269505] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8c2d07b2b000.ost_server_uuid in FULL state after 0 sec [11047.853106] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11062.953810] Lustre: lustre-OST0001: Not available for connect from 192.168.203.16@tcp (stopping) [11063.265854] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11063.273254] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11063.289470] Lustre: Skipped 1 previous similar message [11067.707085] Lustre: server umount lustre-OST0001 complete [11068.041699] LustreError: 324113:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11073.526677] LustreError: 322720:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11073.549433] LustreError: 322720:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11078.643274] LustreError: 322720:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11078.643425] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11078.651566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11078.684667] LustreError: 322720:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11078.742248] Lustre: Skipped 3 previous similar messages [11082.004382] Lustre: server umount lustre-OST0000 complete [11088.869736] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11088.870908] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11088.900748] Lustre: Skipped 1 previous similar message [11088.906953] LustreError: Skipped 1 previous similar message [11093.874189] Lustre: server umount lustre-MDT0000 complete [11093.987177] LustreError: 320899:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11094.018540] LustreError: 320899:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11098.590842] LustreError: 323733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647554 with bad export cookie 13509209338275467623 [11098.601291] LustreError: 323733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11098.607333] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11099.389061] Lustre: server umount lustre-MDT0001 complete [11113.396983] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [11116.559457] Key type lgssc unregistered [11117.024717] LNet: 326425:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11117.047082] LNetError: 326425:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11117.082765] LNet: Removed LNI 192.168.203.116@tcp [11118.429754] Key type .llcrypt unregistered [11118.436983] Key type ._llcrypt unregistered [11148.959078] Key type ._llcrypt registered [11148.961625] Key type .llcrypt registered [11149.069693] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11150.039272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11150.078274] alg: No test for adler32 (adler32-zlib) [11151.352166] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [11151.627419] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [11153.232404] Key type lgssc registered [11154.872071] Lustre: Echo OBD driver; http://www.lustre.org/ [11164.555473] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11164.571443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11166.123822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11171.394599] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11175.407257] LustreError: 327842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11175.431778] LustreError: 327842:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11179.126249] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11180.519909] LustreError: 327843:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11185.636426] LustreError: 327842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11185.693878] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11185.964438] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11189.826786] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11199.785348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11200.301702] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11205.970744] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11210.542855] LustreError: 329522:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11210.588793] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11213.774980] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11215.871303] LustreError: 329524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11215.883310] LustreError: 329524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11220.363572] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11220.629384] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11226.118253] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11228.008822] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11236.301386] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11243.807170] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11244.084732] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11352.048868] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11352.065186] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11352.071186] Lustre: Skipped 1 previous similar message [11355.730628] Lustre: server umount lustre-OST0001 complete [11357.157098] LustreError: 330813:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11357.192488] LustreError: 330813:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11362.280182] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11362.286069] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11362.296337] Lustre: Skipped 2 previous similar messages [11367.450540] Lustre: server umount lustre-OST0000 complete [11372.514569] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11372.516451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11372.534245] Lustre: Skipped 1 previous similar message [11372.547633] Lustre: Skipped 2 previous similar messages [11377.536330] Lustre: server umount lustre-MDT0000 complete [11377.634633] LustreError: 327837:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11377.661934] LustreError: 327837:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [11381.628368] LustreError: 331529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647837 with bad export cookie 17300706039763348087 [11381.658033] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11382.266730] Lustre: server umount lustre-MDT0001 complete [11394.144996] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [11397.427632] Key type lgssc unregistered [11397.713658] LNet: 332786:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11397.726050] LNetError: 332786:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11397.757724] LNet: Removed LNI 192.168.203.116@tcp [11398.879658] Key type .llcrypt unregistered [11398.888294] Key type ._llcrypt unregistered [11450.250689] Key type ._llcrypt registered [11450.253170] Key type .llcrypt registered [11450.367448] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11451.468826] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11451.504711] alg: No test for adler32 (adler32-zlib) [11452.668753] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [11452.886680] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [11454.560350] Key type lgssc registered [11455.847716] Lustre: Echo OBD driver; http://www.lustre.org/ [11465.873833] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11465.891494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11467.450567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11471.798549] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11477.993367] LustreError: 335075:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11478.018358] LustreError: 335075:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11481.114465] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11481.830734] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11488.852662] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11493.475287] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11503.164030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11503.731625] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11506.804627] LustreError: 336562:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11510.261701] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11512.297408] LustreError: 336742:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11513.329975] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11518.444359] LustreError: 336564:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11518.467917] LustreError: 336564:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11518.878177] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11519.236326] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11524.628061] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11526.738027] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11537.655876] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11549.587869] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11563.490974] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11563.501397] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11563.515206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11565.544902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11565.549070] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11565.573786] Lustre: Skipped 2 previous similar messages [11567.535876] Lustre: server umount lustre-MDT0000 complete [11570.658693] LustreError: 335074:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11570.673358] LustreError: 335074:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11570.839376] LustreError: 338144:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788648026 with bad export cookie 3498180007220285051 [11570.842391] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11570.868134] LustreError: 338144:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11571.182710] Lustre: server umount lustre-MDT0001 complete [11585.075906] Lustre: server umount lustre-OST0000 complete [11599.403475] Lustre: server umount lustre-OST0001 complete [11618.495701] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11630.211117] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11630.544212] LustreError: 340341:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11630.637563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11634.828259] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11643.300293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11643.631144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11648.156638] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11651.364706] Lustre: 341481:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11658.344454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11664.612515] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11669.053780] LustreError: 341835:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11669.099987] LustreError: 341835:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11673.292060] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11673.563825] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11673.574590] Lustre: Skipped 1 previous similar message [11675.654196] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11675.665955] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11680.706844] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11689.854224] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11693.875646] Lustre: 343355:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11699.170693] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11699.188827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11699.191479] Lustre: Skipped 2 previous similar messages [11699.202827] Lustre: Skipped 6 previous similar messages [11704.292079] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11704.311832] Lustre: Skipped 3 previous similar messages [11705.073161] Lustre: server umount lustre-MDT0000 complete [11709.250586] LustreError: 340323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788648164 with bad export cookie 3498180007220286752 [11709.262184] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11709.280530] LustreError: 340323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11709.410949] LustreError: 341708:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11709.412120] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11709.433190] LustreError: 341708:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11709.434910] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11709.448179] Lustre: Skipped 2 previous similar messages [11709.473368] Lustre: Skipped 1 previous similar message [11709.959371] Lustre: server umount lustre-MDT0001 complete [11716.869941] Lustre: server umount lustre-OST0000 complete [11721.153518] Lustre: server umount lustre-OST0001 complete [11724.982953] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11726.834221] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11728.467601] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 18:43:01 (1788648181) [11748.786378] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11756.685026] loop0: detected capacity change from 0 to 1099511627776 [11757.456921] SGI XFS with ACLs, security attributes, quota, no debug enabled [11757.468077] *** VALIDATE xfs *** [11757.501900] XFS (loop0): Mounting V5 Filesystem [11757.587218] XFS (loop0): Ending clean mount [11758.902791] loop1: detected capacity change from 0 to 18691697672192 [11863.596464] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [11872.973048] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11906.620386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11906.840880] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11906.864306] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11906.924917] Lustre: lustre-MDT0000: new disk, initializing [11906.987236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11907.004300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11910.984416] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11921.172960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11921.297793] Lustre: 348487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11921.348559] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11921.353671] Lustre: Skipped 1 previous similar message [11921.420462] Lustre: lustre-MDT0001: new disk, initializing [11921.646763] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11921.667876] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11926.573368] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11936.796252] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11942.365193] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11949.073728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11949.361598] Lustre: lustre-OST0000: new disk, initializing [11949.371337] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11949.380636] Lustre: 350456:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11951.090120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11951.112080] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11951.211100] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11954.589104] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11962.361616] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11965.956427] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11966.135311] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11969.282742] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11969.480455] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11971.560936] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11971.565996] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11971.574090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11971.577017] LustreError: Skipped 1 previous similar message [11971.591806] Lustre: Skipped 1 previous similar message [11985.376141] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11985.543914] Lustre: server umount lustre-OST0000 complete [11992.044713] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11992.053988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11992.059239] Lustre: Skipped 2 previous similar messages [11992.060569] Lustre: Skipped 4 previous similar messages [11994.847319] Lustre: server umount lustre-MDT0000 complete [11997.154573] LustreError: 349712:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11997.168101] LustreError: 349712:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11997.731353] LustreError: 348478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788648453 with bad export cookie 3498180007220288306 [11997.742269] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11997.745428] LustreError: 348478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11998.082533] Lustre: server umount lustre-MDT0001 complete [12005.240177] XFS (loop0): Unmounting Filesystem [12009.685523] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 18:47:42 (1788648462) [12017.290796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12018.078349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12018.085967] Lustre: Skipped 2 previous similar messages [12022.760668] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12031.945201] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12036.827761] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12043.892761] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12049.604610] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12055.067010] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12060.565256] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12066.771560] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12078.052885] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12078.053734] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12078.074320] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12078.083349] LustreError: Skipped 1 previous similar message [12078.090166] Lustre: Skipped 2 previous similar messages [12079.873635] Lustre: server umount lustre-OST0000 complete [12089.046481] Lustre: server umount lustre-MDT0000 complete [12092.168959] LustreError: 352996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788648547 with bad export cookie 3498180007220289188 [12092.171192] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12092.194226] LustreError: 352996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12092.559326] Lustre: server umount lustre-MDT0001 complete [12101.843966] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [12103.924476] Key type lgssc unregistered [12104.146210] LNet: 356643:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12104.152815] LNetError: 356643:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12104.163637] LNet: Removed LNI 192.168.203.116@tcp [12104.813420] Key type .llcrypt unregistered [12104.817167] Key type ._llcrypt unregistered [12129.760583] Key type ._llcrypt registered [12129.767051] Key type .llcrypt registered [12129.898888] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [12131.415983] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12131.476309] alg: No test for adler32 (adler32-zlib) [12132.473707] Lustre: Lustre: Build Version: 2.17.58_2_g6837bc1 [12132.765962] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [12134.432145] Key type lgssc registered [12135.536728] Lustre: Echo OBD driver; http://www.lustre.org/ [12143.796387] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12143.810942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12145.417924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12150.183445] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12155.877892] LustreError: 358204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12155.899675] LustreError: 358204:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12159.138755] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12159.455146] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12166.087691] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12175.658613] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12182.118386] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12189.826606] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12190.207304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12196.270640] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12205.748384] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12214.759112] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12214.776701] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12214.790859] Lustre: Skipped 1 previous similar message [12219.037146] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.16@tcp (stopping) [12220.740121] Lustre: server umount lustre-MDT0001 complete [12224.128153] LustreError: 358201:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12229.263945] LustreError: 359639:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12229.305661] LustreError: 359639:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12233.178526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12233.343417] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12233.355481] Lustre: Found index 1 for lustre-MDT0001, updating log [12233.396990] Lustre: 358192:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12233.830158] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12238.734392] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12239.349863] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12240.875952] LustreError: 361779:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12240.881603] LustreError: 361779:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12240.889540] LustreError: 358196:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12240.933899] LustreError: 361788:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.116@tcp: cfg command failed: rc = -17 [12240.940240] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12240.940240] [12249.574387] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12249.600283] Lustre: Skipped 1 previous similar message [12249.609608] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12249.611649] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12249.626015] Lustre: Skipped 3 previous similar messages [12254.693855] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12254.696367] Lustre: Skipped 1 previous similar message [12259.811608] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12259.823140] Lustre: Skipped 2 previous similar messages [12262.368138] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12262.585351] Lustre: server umount lustre-MDT0001 complete [12264.938333] LustreError: 358199:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12264.957103] LustreError: 358199:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12270.058588] LustreError: 358201:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12270.071174] LustreError: 358201:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12274.316941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12274.462760] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12274.469129] Lustre: Found index 1 for lustre-MDT0001, updating log [12274.526942] Lustre: 358193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12275.030465] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12280.314261] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12280.698270] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12282.879187] LustreError: 362848:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12282.889552] LustreError: 362848:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12282.900418] LustreError: 358196:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12282.905361] LustreError: 358196:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12282.981024] LustreError: 362851:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12282.987951] LustreError: 362851:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12282.995294] LustreError: 362851:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.116@tcp: cfg command failed: rc = -17 [12283.010671] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.116@tcp [12283.010671] [12587.501580] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12587.501956] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12587.502156] Lustre: Skipped 1 previous similar message [12587.505506] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12587.560074] Lustre: Skipped 2 previous similar messages [12593.017091] Lustre: server umount lustre-MDT0001 complete [12597.730336] LustreError: 358932:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12597.756116] LustreError: 358932:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12603.050121] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12603.175381] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12603.183773] Lustre: Found index 1 for lustre-MDT0001, updating log [12603.316150] Lustre: 358193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12603.727263] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12609.008666] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12609.419083] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12611.068797] LustreError: 371527:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12611.080269] LustreError: 371527:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12611.106567] LustreError: 358196:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12611.132689] LustreError: 358196:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12611.325102] LustreError: 371535:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12611.350819] LustreError: 371535:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.116@tcp: cfg command failed: rc = -114 [12611.369829] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12611.369829] [12931.556406] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12931.557315] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12931.559127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12931.559621] Lustre: Skipped 4 previous similar messages [12931.580969] Lustre: Skipped 2 previous similar messages [12935.454132] Lustre: server umount lustre-MDT0000 complete [12936.681566] LustreError: 358201:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12936.709800] LustreError: 358201:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12939.489242] LustreError: 358185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788649395 with bad export cookie 9351809981170842659 [12939.501157] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12939.516256] LustreError: 358185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12941.797339] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12941.821505] Lustre: Skipped 1 previous similar message [12946.376133] Lustre: server umount lustre-MDT0001 complete [12951.345413] Lustre: server umount lustre-OST0000 complete [12960.638810] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [12969.242928] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [12996.965213] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 19:04:10 (1788649450) [13002.114504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13002.311442] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13002.346897] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13002.474783] Lustre: lustre-MDT0000: new disk, initializing [13002.557404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13002.574710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13006.102944] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13015.916979] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13015.995461] Lustre: 382350:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13016.049614] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13016.054094] Lustre: Skipped 1 previous similar message [13016.057167] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [13016.060544] Lustre: Skipped 2 previous similar messages [13016.121438] Lustre: lustre-MDT0001: new disk, initializing [13016.199710] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13016.226116] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13016.238292] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13020.358962] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13027.922431] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13032.729771] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13038.356597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13038.589113] Lustre: lustre-OST0000: new disk, initializing [13038.594755] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13038.602393] Lustre: 384315:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13038.684413] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13044.114145] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13047.333336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13047.342351] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13047.413238] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13052.042236] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13061.102601] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13061.109390] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13064.581094] Lustre: server umount lustre-MDT0000 complete [13067.236343] LustreError: 382342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788649522 with bad export cookie 9351809981170844612 [13067.242785] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13067.248559] LustreError: 382342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13067.253031] LustreError: 383160:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13067.482924] Lustre: server umount lustre-MDT0001 complete [13079.670795] Lustre: server umount lustre-OST0000 complete [13113.281555] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [13125.815537] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [13163.763098] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12887 sec ========= 19:06:56 (1788649616) [13165.734365] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 19:06:58 (1788649618) === [13169.696913] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 19:07:02 (1788649622) === [13208.371821] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [13210.758824] Key type lgssc unregistered [13211.024110] LNet: 390678:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13211.030926] LNetError: 390678:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13211.050442] LNet: Removed LNI 192.168.203.116@tcp [13211.772877] Key type .llcrypt unregistered [13211.775429] Key type ._llcrypt unregistered